-- Logs begin at Thu 2022-06-23 07:36:45 UTC, end at Thu 2022-06-23 08:44:58 UTC. -- Jun 23 08:06:17.682647 dv-cinder-847216-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 08:06:17.690891 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 08:06:17 2022] *** Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: nodename: dv-cinder-847216-2-iscsi Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: machine: x86_64 Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: clock source: unix Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: pcre jit disabled Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: detected number of CPU cores: 4 Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: current working directory: / Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: detected binary path: /usr/bin/uwsgi-core Jun 23 08:06:17.692935 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: your processes number limit is 47680 Jun 23 08:06:17.693519 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: your memory page size is 4096 bytes Jun 23 08:06:17.693519 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: detected max file descriptor number: 2048 Jun 23 08:06:17.693519 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: lock engine: pthread robust mutexes Jun 23 08:06:17.693519 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: thunder lock: enabled Jun 23 08:06:17.693519 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 08:06:17.693519 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 08:06:17.736738 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: Python main interpreter initialized at 0x55b3ab5b89c0 Jun 23 08:06:17.736738 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: python threads support enabled Jun 23 08:06:17.736738 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: your server socket listen backlog is limited to 100 connections Jun 23 08:06:17.736738 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: your mercy for graceful operations on workers is 90 seconds Jun 23 08:06:17.737101 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 08:06:17.737165 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: *** Operational MODE: preforking *** Jun 23 08:06:17.737412 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: *** uWSGI is running in multiple interpreter mode *** Jun 23 08:06:17.737412 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: spawned uWSGI master process (pid: 80241) Jun 23 08:06:17.737621 dv-cinder-847216-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 08:06:17.738521 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: spawned uWSGI worker 1 (pid: 80249, cores: 1) Jun 23 08:06:17.738521 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: spawned uWSGI worker 2 (pid: 80250, cores: 1) Jun 23 08:06:17.738691 dv-cinder-847216-2-iscsi devstack@keystone.service[80241]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 08:06:23.148781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 08:06:23.148781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: from cryptography.utils import int_from_bytes Jun 23 08:06:23.149619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 08:06:23.149619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: from cryptography.utils import int_from_bytes Jun 23 08:06:23.173408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 08:06:23.173408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: from cryptography.utils import int_from_bytes Jun 23 08:06:23.173936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 08:06:23.173936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: from cryptography.utils import int_from_bytes Jun 23 08:06:23.197416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 08:06:23.197560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 08:06:23.197788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] command line args: [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 08:06:23.198056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 08:06:23.198187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ================================================================================ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 08:06:23.198476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] admin_token = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.198720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_dir = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.198888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.199105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_source = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.199269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] debug = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.199492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.200837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.200837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.200837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.200837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.200837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.200837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_config_append = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.201206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.201206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_dir = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.201325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.201588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.201823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.202065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.202308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.202489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.202660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.202901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.203144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.203414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.203622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.203868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.204080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.204324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.204566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.204837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.205326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.205508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] publish_errors = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.205741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.205973 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.206151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.206394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.206626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.206859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] standard_threads = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.207087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.207348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.207526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.207702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_journal = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.207880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_json = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.208055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_stderr = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.208230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_syslog = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.208460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.208714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.208950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.209207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.209465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.210029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.210029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.210230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.210465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.210712 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.210959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.211218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.211458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.211704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.211940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.212185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.212429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.212663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.212916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.213168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.213411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.213643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.213892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.214152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.214403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.214651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.214880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.215108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.215398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.215824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.216257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.217509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.217509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.217509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.217509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.217759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.217849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.218099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.218331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.218565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.218800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.219041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.219281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.219514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.219745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.219983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.220112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 08:06:23.220230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.220311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 08:06:23.220462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.220546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] command line args: [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 08:06:23.220698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.220775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 08:06:23.220903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ================================================================================ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 08:06:23.220996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.221151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] admin_token = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.221231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.221384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.221480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] config_dir = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.221565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.221643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.221733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] config_source = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.221846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.221922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] debug = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.222082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.222161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.222317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.222403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.222482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.222554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.222733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.222812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.222975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.223146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.223274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.223362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.223437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_config_append = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.223513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.223585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.223693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.223770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_dir = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.223869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_file = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.223947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.224103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.224179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.224349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.224428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.224589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.224686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.224828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.224911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.224990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.225150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.225417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.225548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.225947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.226024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.226095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.226166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.226246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.226246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.226375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.226445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.226516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.226586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.226664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.226771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.226849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.227010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.227094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.227303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.227398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.227550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.227794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.227870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.227968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] publish_errors = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.228053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.228195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.228273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.228420 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.228497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.228579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.228708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.228824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.228943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.229053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.229178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.229333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] standard_threads = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.229439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.229516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.229677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.229759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.229880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.229964 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.230040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_journal = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.230145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.230224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_json = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.230371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_stderr = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.230446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.230528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_syslog = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.230601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.230758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 08:06:23.230835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.231972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.232051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.232200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.232304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.232438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.232542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.232670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.232772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.232914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233712 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.233959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.234100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.external = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.234179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.234331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.234409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.234578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.234671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.234811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.234898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.235062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.password = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.235150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.235313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.token = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.235404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.235555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.235636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.235791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.235874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.236971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237289 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.237984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.238144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.238220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.238370 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.238494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.238596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.238675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.238837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.238915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.239080 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.239159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.239335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.239418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.239579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.239657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.239805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.239885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.240036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.240113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.240266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.240388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.240477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.240591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.240709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.240985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.241397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.241474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.241560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.241635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.241707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.241779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.241851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.241923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.242973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.243051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.243198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.243296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.243453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.243534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.243691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.243775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.243925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.244935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.245085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.245168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.245318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.245422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.245550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.245638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.245798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.245877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.246027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.246106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.246254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.246332 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.246803 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.246905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.247036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.247113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.247314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.247403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.247546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.247635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.247788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.247869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.248961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.249964 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.250102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.250182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.250331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.250418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.250562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.250645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.250792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.250870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251121 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.251995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252530 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.252989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.253128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.253209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.253353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.253432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.253582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.253660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.253806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.253887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254119 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.254976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.255063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.255241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.255319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.255484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.255563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.255710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.external = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.255790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.255936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.256018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.256165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.256242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.256454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.256723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.257080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.257165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.password = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.257241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.257241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.257372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.token = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.257442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.257658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.257891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.258119 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.258201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.258355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.258443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.258590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.258670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.258825 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.258903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.259057 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.259143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.259324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.259405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.259570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.259649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.259815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.259906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.260060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.260138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.260307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.260386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.260538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.260625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.260776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.260855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.261948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.262985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.263149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.263260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.263409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.263501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.263643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.263724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.263872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.263951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.264112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.264200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.264350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.264429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.264575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.264653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.264807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.264893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.265979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.266057 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.266224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.266314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.266455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.266534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.266695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.266791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.266937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.267969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.268122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.268199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.268372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.268620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.268701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.268865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.268945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.269107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.269193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.269339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.269419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.269566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.269651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.269805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.269893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.270036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.270114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.270279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.270363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.270520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.270613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.270762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.270850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.271004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.271083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.271282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.271426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.271509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.271664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.271743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.271912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.272046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.272228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.272404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.272761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.272919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.272995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.273869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.274949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.275988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.276139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.276215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.276373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.276459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.276617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.276694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.276851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.276928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.277075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.277155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.277299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.277378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.277542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.277620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.277784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.277864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278546 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.278998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.279164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.279325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.279402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.279506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.279646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.279725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.279888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.279970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.280128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.280250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.280358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.280518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.280615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.280763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.280852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.281932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.282953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.283104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.283182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.283359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.283442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.283592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.283684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.283824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.283902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284377 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.284899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 08:06:23.284985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.285211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.285451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.285687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.285918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.286150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.286392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.286629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.286855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.287086 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.287350 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.287641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.287883 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.288112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.288522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.288774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.289008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.289252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.289484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.289720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.289949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] role.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.290190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] role.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.290415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.290659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.290903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.291145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.291417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.291659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.291900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.292143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.292384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.292611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.292853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.293095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.293338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.293578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.293986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.294221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.294468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.294713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.294957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.295238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.295457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.295712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.295961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.296196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.296433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.296663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.296889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.297135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.297386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.297630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.297863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.298096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.298323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.298553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.298785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.299026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.299279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.299530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.299774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.300006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.300244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.300480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.300722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.300953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.301185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.301415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.301653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.301893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.302118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.302354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.302642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.302896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.303134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.303535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.303535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.304019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.304251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.304483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.304713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.304944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.305184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.305421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.305660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.305890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.306132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.306364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.306603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.306828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.307062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.307313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 08:06:23.307481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 08:06:23.315389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.315622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.315851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.324691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.333502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.333733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.333953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.342476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.352783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.353016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.360460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.360695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.496075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.498435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.544878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.545121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.546132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.546334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.609444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.609707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.609844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.609929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.610014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.610092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.610154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.610247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.610332 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.610465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.610614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.610699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.635999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636419 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.636997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.637118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.637204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.637270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.648271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.648518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.660967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.661198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.661323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.661415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.661484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.661662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.661763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.661891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.662027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.662156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.662277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.662398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.662519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.662700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 08:06:23.679675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.679880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 08:06:23.680184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.680311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.680462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 08:06:23.680652 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 08:06:23.680652 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.680852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.680936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.681063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 08:06:23.681184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.681279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.681403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.681538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.681697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.681817 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.691996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.692268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.692499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.692629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.692732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.692842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.693270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.700980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.701818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.704248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.704525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.704804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.705072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.705199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.705350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.705491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.705564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.705642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.705851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.705979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.706097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.706188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.706357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.706449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.706643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.706736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.706871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.707005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.707132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.707433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.707695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.733029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.733749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.734613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.734889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.735353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.735630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.740085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.740472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.740754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.741395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.741395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.741629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.748336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.748560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.748941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.749221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.753026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.753310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.753587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.753715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.753831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.753938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.754112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.754207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.754398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.754524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.754607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.754700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.754878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.754974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.755242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.755529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.773883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.774272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.774398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.774515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.778930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.779076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.782523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.782910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.783039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.783324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.787917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.788518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.791499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.792095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.795097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.795412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.795688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.795812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.795974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.796106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.796225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.796317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.796529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.796657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.796809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.797029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.815348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.815621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.815861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.815995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.816105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.816201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.816368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.816670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.823950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.824674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.825893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.826368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.826637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.829142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.829436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.829733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.829860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.830009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.830142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.830315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.830421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.830588 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.830886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.843426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.843700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.843988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.844113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.844275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.844412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.844605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.844895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.845788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.846071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.846350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.846476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.846698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.846975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.863948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.864640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.867246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.867918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.870508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.871158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.872566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.873206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.874195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.874465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.874736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.875091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.875404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.875674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.881968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.882907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.883575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.884603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.886745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.887028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.887436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.887708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.887834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.888054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.888433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.888703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.896658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.896940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.897361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.897707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.897986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.898259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.905245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.905516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.905791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.905977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 08:06:23.906077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.906282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.906410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.906559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.906654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.906782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.906874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.907082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.907237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.907456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.907662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.907758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.907928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.908060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.908235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.908362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.908489 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.908616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.908741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.908876 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.909002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.909002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.909238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.909532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 08:06:23.947052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b3ab5b89c0 pid: 80249 (default app) Jun 23 08:06:23.947188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b3ab5b89c0 pid: 80250 (default app) Jun 23 08:06:23.950454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b957f5-67f5-4056-a947-17eff164ab26 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:23.950741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b957f5-67f5-4056-a947-17eff164ab26 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:23.951056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b957f5-67f5-4056-a947-17eff164ab26 None None] PATH_INFO: `/v3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:23.955396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Thu Jun 23 08:06:23 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:31.609552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78828a-c164-4af2-92f4-70379a9d7487 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:31.610474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78828a-c164-4af2-92f4-70379a9d7487 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:31.610474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78828a-c164-4af2-92f4-70379a9d7487 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:31.617494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:31.626608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:31.627044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:31.627186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:31.698126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: INFO dbcounter [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] Registered counter for database keystone Jun 23 08:06:31.722577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG oslo_db.sqlalchemy.engines [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 08:06:31.732122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writer thread running {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 08:06:31.917498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG passlib.handlers.bcrypt [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80249) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 08:06:31.917914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG passlib.handlers.bcrypt [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80249) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 08:06:31.973260 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:31.974845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:32.106390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 08:06:32.252859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78828a-c164-4af2-92f4-70379a9d7487 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:32.325802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: INFO dbcounter [None req-ac78828a-c164-4af2-92f4-70379a9d7487 None None] Registered counter for database keystone Jun 23 08:06:32.345998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG oslo_db.sqlalchemy.engines [None req-ac78828a-c164-4af2-92f4-70379a9d7487 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80250) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 08:06:32.355321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writer thread running {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 08:06:32.481316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78828a-c164-4af2-92f4-70379a9d7487 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:32.486700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d6a5c-cfa2-4b60-af97-3955bf8e2bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:32.487408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d6a5c-cfa2-4b60-af97-3955bf8e2bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:32.487698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d6a5c-cfa2-4b60-af97-3955bf8e2bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:32.487979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d6a5c-cfa2-4b60-af97-3955bf8e2bb2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:32.497697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d6a5c-cfa2-4b60-af97-3955bf8e2bb2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:32.550394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING keystone.common.rbac_enforcer.enforcer [None req-157d6a5c-cfa2-4b60-af97-3955bf8e2bb2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 08:06:32.550733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d6a5c-cfa2-4b60-af97-3955bf8e2bb2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:32.562048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Jun 23 08:06:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:06:32.571102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:32.586951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1102c9-89a5-4e1a-81a3-6bb16cc4f9b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:32.591995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899ad3b-5642-4e25-81df-7d15d2fc9ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:32.593199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899ad3b-5642-4e25-81df-7d15d2fc9ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:32.593488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899ad3b-5642-4e25-81df-7d15d2fc9ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:32.593767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899ad3b-5642-4e25-81df-7d15d2fc9ab8 admin admin] PATH_INFO: `/v3/projects/44e27258dd6f419b9b00f5d07551d7c9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:32.597952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899ad3b-5642-4e25-81df-7d15d2fc9ab8 admin admin] RBAC: Authorizing `identity:get_project(project_id=44e27258dd6f419b9b00f5d07551d7c9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:32.649595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f899ad3b-5642-4e25-81df-7d15d2fc9ab8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 08:06:32.650186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899ad3b-5642-4e25-81df-7d15d2fc9ab8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:32.654597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Thu Jun 23 08:06:32 2022] GET /identity/v3/projects/44e27258dd6f419b9b00f5d07551d7c9 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:34.650559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708c8c9-6620-44e5-8602-d87cde23a0f0 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:34.651429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708c8c9-6620-44e5-8602-d87cde23a0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:34.651429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708c8c9-6620-44e5-8602-d87cde23a0f0 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:34.658236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:34.664936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7654e5-b1cf-4b95-a8e3-6913d13966bd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:34.665398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7654e5-b1cf-4b95-a8e3-6913d13966bd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:34.665491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7654e5-b1cf-4b95-a8e3-6913d13966bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:34.690729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0c7654e5-b1cf-4b95-a8e3-6913d13966bd None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:34.692287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0c7654e5-b1cf-4b95-a8e3-6913d13966bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:34.701643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 08:06:34.727985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708c8c9-6620-44e5-8602-d87cde23a0f0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:34.742398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708c8c9-6620-44e5-8602-d87cde23a0f0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:34.748373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ecd69-4b3e-4814-bb7d-9a46ba58abae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:34.749017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ecd69-4b3e-4814-bb7d-9a46ba58abae admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:34.749301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ecd69-4b3e-4814-bb7d-9a46ba58abae admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:34.749581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ecd69-4b3e-4814-bb7d-9a46ba58abae admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:34.758366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ecd69-4b3e-4814-bb7d-9a46ba58abae admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:34.760122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ecd69-4b3e-4814-bb7d-9a46ba58abae admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:34.769708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Jun 23 08:06:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:06:34.776088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7654e5-b1cf-4b95-a8e3-6913d13966bd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:34.790734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7654e5-b1cf-4b95-a8e3-6913d13966bd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:34.798109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e79fcb-5ff9-4bf4-9237-cdcda60cc52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:34.798978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e79fcb-5ff9-4bf4-9237-cdcda60cc52a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:34.799263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e79fcb-5ff9-4bf4-9237-cdcda60cc52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:34.799555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e79fcb-5ff9-4bf4-9237-cdcda60cc52a admin admin] PATH_INFO: `/v3/users/76b6ddf40257432aa78fadccd4348cd5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:34.800991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e79fcb-5ff9-4bf4-9237-cdcda60cc52a admin admin] RBAC: Authorizing `identity:get_user(user_id=76b6ddf40257432aa78fadccd4348cd5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:34.802828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e79fcb-5ff9-4bf4-9237-cdcda60cc52a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:34.804492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Thu Jun 23 08:06:34 2022] GET /identity/v3/users/76b6ddf40257432aa78fadccd4348cd5 => generated 247 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:36.855657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bce89-115e-493c-8a4a-61441400539e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:36.856170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bce89-115e-493c-8a4a-61441400539e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:36.856170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bce89-115e-493c-8a4a-61441400539e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:36.857451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:36.862824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5b6a3-2225-4a23-929e-7403afe6e4fa None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:36.863265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5b6a3-2225-4a23-929e-7403afe6e4fa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:36.863424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5b6a3-2225-4a23-929e-7403afe6e4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:36.880785 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-73f5b6a3-2225-4a23-929e-7403afe6e4fa None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:36.882092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-73f5b6a3-2225-4a23-929e-7403afe6e4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:36.967818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 6/11] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:36.970740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6a99e-1abf-4dff-ba51-285bdbe8ff16 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:36.971026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6a99e-1abf-4dff-ba51-285bdbe8ff16 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:36.971333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6a99e-1abf-4dff-ba51-285bdbe8ff16 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:36.972725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 6/12] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:36.978026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e202b7-a9d6-4c87-92a9-089d58220393 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:36.978320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e202b7-a9d6-4c87-92a9-089d58220393 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:36.978598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e202b7-a9d6-4c87-92a9-089d58220393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:36.991387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6a99e-1abf-4dff-ba51-285bdbe8ff16 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:36.996571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e7e202b7-a9d6-4c87-92a9-089d58220393 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:36.998084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e7e202b7-a9d6-4c87-92a9-089d58220393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:37.015373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:37.069333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:37.090933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e202b7-a9d6-4c87-92a9-089d58220393 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:37.137583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6a99e-1abf-4dff-ba51-285bdbe8ff16 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:37.171622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a8fc6-6121-4b4e-a2cc-7cdf4a7be8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:37.172445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a8fc6-6121-4b4e-a2cc-7cdf4a7be8ef None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:37.172728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a8fc6-6121-4b4e-a2cc-7cdf4a7be8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:37.173011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a8fc6-6121-4b4e-a2cc-7cdf4a7be8ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:37.185862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a8fc6-6121-4b4e-a2cc-7cdf4a7be8ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:37.194538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e202b7-a9d6-4c87-92a9-089d58220393 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a00a8fc6-6121-4b4e-a2cc-7cdf4a7be8ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:37.197166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:37.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:37.199630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:37.204122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a8fc6-6121-4b4e-a2cc-7cdf4a7be8ef None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:37.230727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bb0e5-f7bd-4b22-9898-e9645d3a25d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:37.231425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bb0e5-f7bd-4b22-9898-e9645d3a25d4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:37.231711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bb0e5-f7bd-4b22-9898-e9645d3a25d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:37.231992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bb0e5-f7bd-4b22-9898-e9645d3a25d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:37.240607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:37.255192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:37.302469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bb0e5-f7bd-4b22-9898-e9645d3a25d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:37.304575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bb0e5-f7bd-4b22-9898-e9645d3a25d4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:37.312472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a8fc6-6121-4b4e-a2cc-7cdf4a7be8ef None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:37.346290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:37.346967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:37.347270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:37.347543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:37.348295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:37.348609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:37.350051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:37.361990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:06:37.367646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:37.406452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 08:06:37.412043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bb0e5-f7bd-4b22-9898-e9645d3a25d4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:37.469507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:37.472908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:37.476525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e3b40-0473-4883-904b-290039af682c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:37.512056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eed59-7492-44ce-a709-dcac0cf73e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:37.512953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eed59-7492-44ce-a709-dcac0cf73e8b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:37.513238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eed59-7492-44ce-a709-dcac0cf73e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:37.513519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eed59-7492-44ce-a709-dcac0cf73e8b None admin] PATH_INFO: `/v3/users/76b6ddf40257432aa78fadccd4348cd5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:37.515177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eed59-7492-44ce-a709-dcac0cf73e8b None admin] RBAC: Authorizing `identity:get_user(user_id=76b6ddf40257432aa78fadccd4348cd5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:37.516678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eed59-7492-44ce-a709-dcac0cf73e8b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:37.518321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:37 2022] GET /identity/v3/users/76b6ddf40257432aa78fadccd4348cd5 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:37.520644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bb0e5-f7bd-4b22-9898-e9645d3a25d4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:37.523687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eed59-7492-44ce-a709-dcac0cf73e8b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:37.556317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:37.557118 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:37.557407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:37.557800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:37.568840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:37.570326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:06:37.591422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:37.592442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:37.593345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:06:37.593345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:06:37.593345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:06:37.593345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:06:37.593345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:37.598244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:37.643734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eed59-7492-44ce-a709-dcac0cf73e8b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:37.678417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf1ea-0d7c-4b6d-8a4f-59807fed7e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:37.679372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf1ea-0d7c-4b6d-8a4f-59807fed7e69 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:37.679660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf1ea-0d7c-4b6d-8a4f-59807fed7e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:37.679940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf1ea-0d7c-4b6d-8a4f-59807fed7e69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:37.681244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cf1ea-0d7c-4b6d-8a4f-59807fed7e69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:37.682699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cf1ea-0d7c-4b6d-8a4f-59807fed7e69 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:37.684332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:06:37.690019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf1ea-0d7c-4b6d-8a4f-59807fed7e69 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:37.702586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af7705-94d6-4717-ac54-58702a652799 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:37.704942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:37.721411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:37.737235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395029e-3d8e-4a19-b6c2-c1e1a920c2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:37.737909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395029e-3d8e-4a19-b6c2-c1e1a920c2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:37.738184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395029e-3d8e-4a19-b6c2-c1e1a920c2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:37.738456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395029e-3d8e-4a19-b6c2-c1e1a920c2c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:37.739180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395029e-3d8e-4a19-b6c2-c1e1a920c2c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:37.739514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395029e-3d8e-4a19-b6c2-c1e1a920c2c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:37.741181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395029e-3d8e-4a19-b6c2-c1e1a920c2c7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:37.758210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:06:37.798119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf1ea-0d7c-4b6d-8a4f-59807fed7e69 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:37.838598 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7d97b-4c4b-4607-bc25-305108462589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:37.839305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d97b-4c4b-4607-bc25-305108462589 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:37.839589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d97b-4c4b-4607-bc25-305108462589 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:37.839866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d97b-4c4b-4607-bc25-305108462589 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:37.840728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7d97b-4c4b-4607-bc25-305108462589 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.domain.id=default, user.id=76b6ddf40257432aa78fadccd4348cd5) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:37.841047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7d97b-4c4b-4607-bc25-305108462589 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:37.842509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7d97b-4c4b-4607-bc25-305108462589 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:37.852917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Jun 23 08:06:37 2022] GET /identity/v3/role_assignments?user.id=76b6ddf40257432aa78fadccd4348cd5&scope.domain.id=default&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:06:40.093695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069660-fdec-471c-9cff-98300c0843b2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.094729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069660-fdec-471c-9cff-98300c0843b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.095221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069660-fdec-471c-9cff-98300c0843b2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.096784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:40.102045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.102869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.103382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.106369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1877c08-601c-4f3d-9284-599386354a30 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.107257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1877c08-601c-4f3d-9284-599386354a30 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.108238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1877c08-601c-4f3d-9284-599386354a30 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.111024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 12/23] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:40.117026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb7d5-b00d-4d5a-8c64-b9dd6826e7f3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.117555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb7d5-b00d-4d5a-8c64-b9dd6826e7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.123176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb7d5-b00d-4d5a-8c64-b9dd6826e7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.130861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG passlib.handlers.bcrypt [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80250) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 08:06:40.132584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG passlib.handlers.bcrypt [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80250) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 08:06:40.153791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f5dbb7d5-b00d-4d5a-8c64-b9dd6826e7f3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:40.155311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f5dbb7d5-b00d-4d5a-8c64-b9dd6826e7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:40.219094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:40.222182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:40.320757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 13/24] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:06:40.361762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb7d5-b00d-4d5a-8c64-b9dd6826e7f3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:40.373595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 12/25] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:40.410300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:40.463937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:40.567616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4581f-c952-420e-ae01-e1d19f03ac27 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:40.604842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb7d5-b00d-4d5a-8c64-b9dd6826e7f3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:40.613868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:40.614970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:40.616124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.616607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.616944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.639638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:40.641193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:06:40.660775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:40.661664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:40.662437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:06:40.662437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:06:40.662437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:06:40.662437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:40.662437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 13/26] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:06:40.668448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:40.672133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6968ed-ed9d-41f4-b3d0-eb5cd5dd8e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:40.672957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6968ed-ed9d-41f4-b3d0-eb5cd5dd8e36 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.673236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6968ed-ed9d-41f4-b3d0-eb5cd5dd8e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.673507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6968ed-ed9d-41f4-b3d0-eb5cd5dd8e36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.699390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6968ed-ed9d-41f4-b3d0-eb5cd5dd8e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ea6968ed-ed9d-41f4-b3d0-eb5cd5dd8e36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:40.709000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:40.710140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:40.710912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:06:40.725278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 14/27] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:06:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:06:40.726246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6968ed-ed9d-41f4-b3d0-eb5cd5dd8e36 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:40.812638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:40.838260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb291c-f30b-45fd-9175-e8e8ebb1cd65 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:40.876330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015f915-8cdb-4962-857b-e5c25b1cf9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:40.877005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f915-8cdb-4962-857b-e5c25b1cf9dc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.877288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f915-8cdb-4962-857b-e5c25b1cf9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.877562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f915-8cdb-4962-857b-e5c25b1cf9dc None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.878302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015f915-8cdb-4962-857b-e5c25b1cf9dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:40.878611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015f915-8cdb-4962-857b-e5c25b1cf9dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:40.880086 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015f915-8cdb-4962-857b-e5c25b1cf9dc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:40.899420 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 14/28] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:06:40.900786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfafa1-cc2b-4168-811c-9f6c0089b108 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.901073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfafa1-cc2b-4168-811c-9f6c0089b108 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.901343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfafa1-cc2b-4168-811c-9f6c0089b108 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.907465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 15/29] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:40.909060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b45f62-652b-4c46-bef4-571e09dd93ca None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.909359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b45f62-652b-4c46-bef4-571e09dd93ca None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.909834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b45f62-652b-4c46-bef4-571e09dd93ca None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.912290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 16/30] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:40.913104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b45f62-652b-4c46-bef4-571e09dd93ca None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:40.928261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6968ed-ed9d-41f4-b3d0-eb5cd5dd8e36 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:40.983609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ce7bc-b8ac-4951-bdf3-c4fd3d491ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:40.984305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce7bc-b8ac-4951-bdf3-c4fd3d491ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:40.984593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce7bc-b8ac-4951-bdf3-c4fd3d491ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:40.984872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce7bc-b8ac-4951-bdf3-c4fd3d491ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:40.985636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ce7bc-b8ac-4951-bdf3-c4fd3d491ca0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:40.985949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ce7bc-b8ac-4951-bdf3-c4fd3d491ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:40.987511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ce7bc-b8ac-4951-bdf3-c4fd3d491ca0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:41.015888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 15/31] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:41.019520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc108b5-c247-48c6-b95d-56fb0beac367 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.020034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc108b5-c247-48c6-b95d-56fb0beac367 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.020670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc108b5-c247-48c6-b95d-56fb0beac367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.044620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7bc108b5-c247-48c6-b95d-56fb0beac367 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:41.045783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b45f62-652b-4c46-bef4-571e09dd93ca None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:41.047775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7bc108b5-c247-48c6-b95d-56fb0beac367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:41.062388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:41.096520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e944f3-0670-49a5-9f6d-417dc0f61d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:41.097970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e944f3-0670-49a5-9f6d-417dc0f61d9f None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.097970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e944f3-0670-49a5-9f6d-417dc0f61d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.097970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e944f3-0670-49a5-9f6d-417dc0f61d9f None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.099138 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e944f3-0670-49a5-9f6d-417dc0f61d9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:41.101654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e944f3-0670-49a5-9f6d-417dc0f61d9f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:41.113964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:41.145662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 16/32] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:41.148022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd98a7-451e-4bbf-9920-7a9cbab43b3c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.148022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd98a7-451e-4bbf-9920-7a9cbab43b3c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.148022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd98a7-451e-4bbf-9920-7a9cbab43b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.181460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-76dd98a7-451e-4bbf-9920-7a9cbab43b3c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:41.182682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76dd98a7-451e-4bbf-9920-7a9cbab43b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:41.284052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 17/33] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:41.285264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd98a7-451e-4bbf-9920-7a9cbab43b3c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:41.304568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 17/34] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:06:40 2022] POST /identity/v3/projects => generated 301 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:06:41.311407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e944f3-0670-49a5-9f6d-417dc0f61d9f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:41.447308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:41.502367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd98a7-451e-4bbf-9920-7a9cbab43b3c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:41.543856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e944f3-0670-49a5-9f6d-417dc0f61d9f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:41.569689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3b180-a958-4484-9236-5d82ba130bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:41.570601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3b180-a958-4484-9236-5d82ba130bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.570879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3b180-a958-4484-9236-5d82ba130bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.571148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3b180-a958-4484-9236-5d82ba130bf5 None admin] PATH_INFO: `/v3/users/76b6ddf40257432aa78fadccd4348cd5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.574051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3b180-a958-4484-9236-5d82ba130bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=76b6ddf40257432aa78fadccd4348cd5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:41.574970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3b180-a958-4484-9236-5d82ba130bf5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:41.577793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 18/35] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:41 2022] GET /identity/v3/users/76b6ddf40257432aa78fadccd4348cd5 => generated 247 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:41.579503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3b180-a958-4484-9236-5d82ba130bf5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:41.612662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6ab96-24a0-4f1b-be3c-8c2eace86040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:41.613380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ab96-24a0-4f1b-be3c-8c2eace86040 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.613667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ab96-24a0-4f1b-be3c-8c2eace86040 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.613937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ab96-24a0-4f1b-be3c-8c2eace86040 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.614711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6ab96-24a0-4f1b-be3c-8c2eace86040 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:41.617022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6ab96-24a0-4f1b-be3c-8c2eace86040 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:41.634828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 18/36] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 08:06:41 2022] POST /identity/v3/roles => generated 225 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:06:41.636837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3ea4-2631-48fd-8aa9-b3fa82c2b1b1 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.637130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3ea4-2631-48fd-8aa9-b3fa82c2b1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.637410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3ea4-2631-48fd-8aa9-b3fa82c2b1b1 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.639515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 19/37] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:41.640339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3ea4-2631-48fd-8aa9-b3fa82c2b1b1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:41.717182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:41.722804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3b180-a958-4484-9236-5d82ba130bf5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:41.764641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb21cf3-94c9-4cc4-a9e5-f7cd3d78cb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:41.765294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21cf3-94c9-4cc4-a9e5-f7cd3d78cb39 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.765598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21cf3-94c9-4cc4-a9e5-f7cd3d78cb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.765871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21cf3-94c9-4cc4-a9e5-f7cd3d78cb39 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.766618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb21cf3-94c9-4cc4-a9e5-f7cd3d78cb39 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:41.768071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb21cf3-94c9-4cc4-a9e5-f7cd3d78cb39 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:41.777936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3ea4-2631-48fd-8aa9-b3fa82c2b1b1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:41.822713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ff055-a2f7-4481-874c-8c93fccd4aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:41.823586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ff055-a2f7-4481-874c-8c93fccd4aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.823869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ff055-a2f7-4481-874c-8c93fccd4aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.824132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ff055-a2f7-4481-874c-8c93fccd4aa8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.826304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ff055-a2f7-4481-874c-8c93fccd4aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:41.827904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ff055-a2f7-4481-874c-8c93fccd4aa8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:41.829745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 20/38] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:41.830976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d92e82-045f-48f2-a873-c6b567bb02fe None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.831168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d92e82-045f-48f2-a873-c6b567bb02fe None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.831477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d92e82-045f-48f2-a873-c6b567bb02fe None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.832726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 21/39] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:41.835057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a790bed-bab7-4ec1-bc3f-9cf64e734261 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.835390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a790bed-bab7-4ec1-bc3f-9cf64e734261 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.835656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a790bed-bab7-4ec1-bc3f-9cf64e734261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.854399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6a790bed-bab7-4ec1-bc3f-9cf64e734261 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:41.855942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6a790bed-bab7-4ec1-bc3f-9cf64e734261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:41.888243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 19/40] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 08:06:41 2022] POST /identity/v3/roles => generated 219 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:06:41.889520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89160a4-72a3-4c96-8b1b-c65bc5c657d3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:41.889823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89160a4-72a3-4c96-8b1b-c65bc5c657d3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:41.890086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89160a4-72a3-4c96-8b1b-c65bc5c657d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:41.897843 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:41.909862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b89160a4-72a3-4c96-8b1b-c65bc5c657d3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:41.911560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b89160a4-72a3-4c96-8b1b-c65bc5c657d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:41.955234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 22/41] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:41.955234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a790bed-bab7-4ec1-bc3f-9cf64e734261 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:42.000395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 20/42] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:42.000993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89160a4-72a3-4c96-8b1b-c65bc5c657d3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:42.127094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89160a4-72a3-4c96-8b1b-c65bc5c657d3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:42.131287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:42.186521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a790bed-bab7-4ec1-bc3f-9cf64e734261 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:42.224670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4382fd2-861f-4cdb-aba9-8f0e41397b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:42.225595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382fd2-861f-4cdb-aba9-8f0e41397b5a None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:42.225877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382fd2-861f-4cdb-aba9-8f0e41397b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:42.226148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382fd2-861f-4cdb-aba9-8f0e41397b5a None admin] PATH_INFO: `/v3/domains/default/users/76b6ddf40257432aa78fadccd4348cd5/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:42.228598 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4382fd2-861f-4cdb-aba9-8f0e41397b5a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=76b6ddf40257432aa78fadccd4348cd5, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:42.230135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4382fd2-861f-4cdb-aba9-8f0e41397b5a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:42.246567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 23/43] 10.222.0.33 () {58 vars in 1361 bytes} [Thu Jun 23 08:06:41 2022] PUT /identity/v3/domains/default/users/76b6ddf40257432aa78fadccd4348cd5/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:06:42.247863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4382fd2-861f-4cdb-aba9-8f0e41397b5a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:42.274519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dac820-78f0-4f6b-a9af-98b2c6ed26df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:42.276010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac820-78f0-4f6b-a9af-98b2c6ed26df None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:42.276463 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac820-78f0-4f6b-a9af-98b2c6ed26df None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:42.276873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac820-78f0-4f6b-a9af-98b2c6ed26df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:42.278454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dac820-78f0-4f6b-a9af-98b2c6ed26df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:42.280189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dac820-78f0-4f6b-a9af-98b2c6ed26df None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:42.282126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 21/44] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:42.287767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dac820-78f0-4f6b-a9af-98b2c6ed26df None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:42.291249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:42.369740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4382fd2-861f-4cdb-aba9-8f0e41397b5a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:42.380899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:42.403881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dac820-78f0-4f6b-a9af-98b2c6ed26df None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:42.417269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602b38c-5326-4ac5-b9e3-76009be54a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:42.418248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602b38c-5326-4ac5-b9e3-76009be54a45 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:42.418538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602b38c-5326-4ac5-b9e3-76009be54a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:42.418821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602b38c-5326-4ac5-b9e3-76009be54a45 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:42.419633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602b38c-5326-4ac5-b9e3-76009be54a45 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:42.421020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602b38c-5326-4ac5-b9e3-76009be54a45 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:42.436176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 24/45] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 08:06:42 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:06:42.439707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78e85d-98b2-4dae-9700-2920c2fbde4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:42.440381 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e85d-98b2-4dae-9700-2920c2fbde4d None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:42.440665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e85d-98b2-4dae-9700-2920c2fbde4d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:42.440941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78e85d-98b2-4dae-9700-2920c2fbde4d None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:42.441670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78e85d-98b2-4dae-9700-2920c2fbde4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:42.443182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78e85d-98b2-4dae-9700-2920c2fbde4d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:42.475385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 22/46] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:06:42 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:06:44.250307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09969342-5f59-4473-8bb1-7914326be9b4 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.250857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09969342-5f59-4473-8bb1-7914326be9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.250857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09969342-5f59-4473-8bb1-7914326be9b4 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.252278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 25/47] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:44.257723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3584-4998-4831-a56b-5d3d1f46ee1d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.258167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3584-4998-4831-a56b-5d3d1f46ee1d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.258305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3584-4998-4831-a56b-5d3d1f46ee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.276167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7ffc3584-4998-4831-a56b-5d3d1f46ee1d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:44.277388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ffc3584-4998-4831-a56b-5d3d1f46ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:44.331442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:44.351473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 23/48] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:44.375179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09969342-5f59-4473-8bb1-7914326be9b4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:44.485322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09969342-5f59-4473-8bb1-7914326be9b4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:44.506208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649a803-582b-427b-9a27-aa6ceb897f30 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.506498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649a803-582b-427b-9a27-aa6ceb897f30 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.506757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649a803-582b-427b-9a27-aa6ceb897f30 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.508282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 24/49] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:44.513283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd1c8-2aab-41f5-b20e-0ce617144f99 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.513549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd1c8-2aab-41f5-b20e-0ce617144f99 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.513834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd1c8-2aab-41f5-b20e-0ce617144f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.523477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fadf7-65c4-4437-990d-fa055ec07020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:44.524362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fadf7-65c4-4437-990d-fa055ec07020 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.524596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fadf7-65c4-4437-990d-fa055ec07020 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.524871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fadf7-65c4-4437-990d-fa055ec07020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.531114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ed9cd1c8-2aab-41f5-b20e-0ce617144f99 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:44.532346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ed9cd1c8-2aab-41f5-b20e-0ce617144f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:44.535180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fadf7-65c4-4437-990d-fa055ec07020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-042fadf7-65c4-4437-990d-fa055ec07020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:44.543268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:44.544205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:44.544945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 26/50] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:06:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:44.550250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fadf7-65c4-4437-990d-fa055ec07020 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:44.577707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:44.601902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 25/51] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:06:44.625387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cd1c8-2aab-41f5-b20e-0ce617144f99 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:44.652441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fadf7-65c4-4437-990d-fa055ec07020 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:44.685903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:44.686574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.686852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.687123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.688174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:44.688479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:44.689871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:44.698179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 27/52] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:06:44.703661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:44.724384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cd1c8-2aab-41f5-b20e-0ce617144f99 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:44.738337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:44.758680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3f20-ed5d-46ee-aed5-1053620be8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:44.759444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3f20-ed5d-46ee-aed5-1053620be8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.759730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3f20-ed5d-46ee-aed5-1053620be8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.760030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3f20-ed5d-46ee-aed5-1053620be8a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.761236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa3f20-ed5d-46ee-aed5-1053620be8a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:44.762733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa3f20-ed5d-46ee-aed5-1053620be8a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:44.764538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 26/53] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:44.770031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3f20-ed5d-46ee-aed5-1053620be8a8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:44.792953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:44.802616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb22307-6faf-4c04-8a9e-22e08a786b4a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:44.837682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdd1db-9773-4c47-a399-9bf798d68c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:44.838616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdd1db-9773-4c47-a399-9bf798d68c9c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.838898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdd1db-9773-4c47-a399-9bf798d68c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.839172 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdd1db-9773-4c47-a399-9bf798d68c9c None admin] PATH_INFO: `/v3/users/76b6ddf40257432aa78fadccd4348cd5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.840427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdd1db-9773-4c47-a399-9bf798d68c9c None admin] RBAC: Authorizing `identity:get_user(user_id=76b6ddf40257432aa78fadccd4348cd5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:44.841904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdd1db-9773-4c47-a399-9bf798d68c9c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:44.843590 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 28/54] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:44 2022] GET /identity/v3/users/76b6ddf40257432aa78fadccd4348cd5 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:44.848995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdd1db-9773-4c47-a399-9bf798d68c9c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:44.871173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3f20-ed5d-46ee-aed5-1053620be8a8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:44.907660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d608-b24c-419a-94fa-e5e4a47aa8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:44.908328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d608-b24c-419a-94fa-e5e4a47aa8f0 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.908608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d608-b24c-419a-94fa-e5e4a47aa8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.908882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d608-b24c-419a-94fa-e5e4a47aa8f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.909587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d608-b24c-419a-94fa-e5e4a47aa8f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:44.911070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d608-b24c-419a-94fa-e5e4a47aa8f0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:44.935683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 27/55] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Jun 23 08:06:44 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:06:44.949170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdd1db-9773-4c47-a399-9bf798d68c9c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:44.987329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89efa20-fd47-47b2-84f4-cea7c3b1b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:44.988155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89efa20-fd47-47b2-84f4-cea7c3b1b37e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:44.988440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89efa20-fd47-47b2-84f4-cea7c3b1b37e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:44.988715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89efa20-fd47-47b2-84f4-cea7c3b1b37e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:44.989969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89efa20-fd47-47b2-84f4-cea7c3b1b37e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:44.991580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89efa20-fd47-47b2-84f4-cea7c3b1b37e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:44.993541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 29/56] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:06:44.999812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d608-b24c-419a-94fa-e5e4a47aa8f0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:45.038398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:45.112918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d608-b24c-419a-94fa-e5e4a47aa8f0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:45.147377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80024075-93d4-4dd1-a1db-8dc5992a2431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:45.148046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80024075-93d4-4dd1-a1db-8dc5992a2431 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:45.148329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80024075-93d4-4dd1-a1db-8dc5992a2431 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:45.148612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80024075-93d4-4dd1-a1db-8dc5992a2431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:45.149451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80024075-93d4-4dd1-a1db-8dc5992a2431 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.domain.id=default, user.id=76b6ddf40257432aa78fadccd4348cd5) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:45.149762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80024075-93d4-4dd1-a1db-8dc5992a2431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:45.151241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80024075-93d4-4dd1-a1db-8dc5992a2431 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:45.158571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 28/57] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Jun 23 08:06:44 2022] GET /identity/v3/role_assignments?user.id=76b6ddf40257432aa78fadccd4348cd5&scope.domain.id=default&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:46.958215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5138c6-fb10-4317-9d2b-035264558911 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:46.958764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5138c6-fb10-4317-9d2b-035264558911 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:46.958764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5138c6-fb10-4317-9d2b-035264558911 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:46.960252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 30/58] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:46.966413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21397c81-0329-4af9-9975-a6c064a0f46f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:46.966840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21397c81-0329-4af9-9975-a6c064a0f46f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:46.967017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21397c81-0329-4af9-9975-a6c064a0f46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:46.993394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-21397c81-0329-4af9-9975-a6c064a0f46f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:46.994578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-21397c81-0329-4af9-9975-a6c064a0f46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:47.065045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 29/59] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:47.089145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5138c6-fb10-4317-9d2b-035264558911 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:47.120376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:47.200579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5138c6-fb10-4317-9d2b-035264558911 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:47.235608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aaf8d-318e-41dc-897a-21d1cb160abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:47.236335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aaf8d-318e-41dc-897a-21d1cb160abe None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:47.236618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aaf8d-318e-41dc-897a-21d1cb160abe None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:47.236889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aaf8d-318e-41dc-897a-21d1cb160abe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:47.238184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aaf8d-318e-41dc-897a-21d1cb160abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:47.239687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aaf8d-318e-41dc-897a-21d1cb160abe None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:47.241356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 31/60] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:47.247540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21397c81-0329-4af9-9975-a6c064a0f46f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:47.276143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:47.355835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21397c81-0329-4af9-9975-a6c064a0f46f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:47.391130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c52d02-b81d-4f6e-8dad-24eba8a19aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:47.391839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52d02-b81d-4f6e-8dad-24eba8a19aa2 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:47.392118 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52d02-b81d-4f6e-8dad-24eba8a19aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:47.392405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52d02-b81d-4f6e-8dad-24eba8a19aa2 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:47.393087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c52d02-b81d-4f6e-8dad-24eba8a19aa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:47.394536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c52d02-b81d-4f6e-8dad-24eba8a19aa2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:47.421905 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 30/61] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:06:47 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:06:50.540830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7d075-3b34-4125-b8d4-964801a98741 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:50.541801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7d075-3b34-4125-b8d4-964801a98741 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:50.542297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7d075-3b34-4125-b8d4-964801a98741 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:50.543904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 32/62] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:50.549734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea58a7f-e69a-47f0-af1d-a5a9c3e7de82 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:50.550505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea58a7f-e69a-47f0-af1d-a5a9c3e7de82 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:50.550932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea58a7f-e69a-47f0-af1d-a5a9c3e7de82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:50.576313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5ea58a7f-e69a-47f0-af1d-a5a9c3e7de82 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:50.577786 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5ea58a7f-e69a-47f0-af1d-a5a9c3e7de82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:50.672629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 31/63] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:50.699085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb305cc-7742-482a-9538-b80f07bcf0c2 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:50.699772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb305cc-7742-482a-9538-b80f07bcf0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:50.700262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb305cc-7742-482a-9538-b80f07bcf0c2 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:50.701826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 33/64] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:50.706955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c2564-b0c2-4b8c-88ce-33b1333002be None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:50.707284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c2564-b0c2-4b8c-88ce-33b1333002be None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:50.707571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c2564-b0c2-4b8c-88ce-33b1333002be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:50.708789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb305cc-7742-482a-9538-b80f07bcf0c2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:50.724699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-193c2564-b0c2-4b8c-88ce-33b1333002be None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:50.726004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-193c2564-b0c2-4b8c-88ce-33b1333002be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:50.735856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:50.808965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 32/65] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:50.810686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afacfc2-8682-4ab6-b779-4f8db0416e73 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:50.811191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afacfc2-8682-4ab6-b779-4f8db0416e73 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:50.811660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afacfc2-8682-4ab6-b779-4f8db0416e73 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:50.813103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 33/66] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:50.818362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62945e-3713-41b5-928a-4dbce4cbd59d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:50.818987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62945e-3713-41b5-928a-4dbce4cbd59d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:50.819492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62945e-3713-41b5-928a-4dbce4cbd59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:50.840848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc62945e-3713-41b5-928a-4dbce4cbd59d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:50.843040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc62945e-3713-41b5-928a-4dbce4cbd59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:50.859132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:50.870380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb305cc-7742-482a-9538-b80f07bcf0c2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:50.913611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6172b01-c694-4d71-9a3c-351f19ee7fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:50.914899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6172b01-c694-4d71-9a3c-351f19ee7fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:50.915501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6172b01-c694-4d71-9a3c-351f19ee7fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:50.915921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6172b01-c694-4d71-9a3c-351f19ee7fd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:50.927574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 34/67] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:06:50.932396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62945e-3713-41b5-928a-4dbce4cbd59d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:50.935418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6172b01-c694-4d71-9a3c-351f19ee7fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f6172b01-c694-4d71-9a3c-351f19ee7fd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:50.950013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:50.950994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:50.951787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 34/68] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:06:50.955765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6172b01-c694-4d71-9a3c-351f19ee7fd3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.023888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.055265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62945e-3713-41b5-928a-4dbce4cbd59d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.066436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6172b01-c694-4d71-9a3c-351f19ee7fd3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.090698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31270a49-8423-4e33-8e9e-dd1b85e32086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.091544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31270a49-8423-4e33-8e9e-dd1b85e32086 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.091824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31270a49-8423-4e33-8e9e-dd1b85e32086 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.092094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31270a49-8423-4e33-8e9e-dd1b85e32086 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.102511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31270a49-8423-4e33-8e9e-dd1b85e32086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:51.102875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e3fb0-2bb1-47c0-8460-5f40139dcb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.103717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3fb0-2bb1-47c0-8460-5f40139dcb5d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.104003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3fb0-2bb1-47c0-8460-5f40139dcb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.104279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3fb0-2bb1-47c0-8460-5f40139dcb5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-31270a49-8423-4e33-8e9e-dd1b85e32086 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:51.110658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:51.111577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:06:51.112333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 35/69] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:06:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:51.112945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31270a49-8423-4e33-8e9e-dd1b85e32086 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.120719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e3fb0-2bb1-47c0-8460-5f40139dcb5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-608e3fb0-2bb1-47c0-8460-5f40139dcb5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:51.128838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:51.129885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:51.130672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 35/70] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:06:51.131627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f9ba6-09fa-4986-b0e6-ec5ac89d3270 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.131732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f9ba6-09fa-4986-b0e6-ec5ac89d3270 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.132415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f9ba6-09fa-4986-b0e6-ec5ac89d3270 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.137397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 36/71] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:51.137397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f9ba6-09fa-4986-b0e6-ec5ac89d3270 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.151426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.230619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31270a49-8423-4e33-8e9e-dd1b85e32086 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.242801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f9ba6-09fa-4986-b0e6-ec5ac89d3270 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.266958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.267673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.267960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.268239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.269107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:51.269528 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:51.270941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:51.272678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.277886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1410db1-1f6f-43eb-a670-3f03788329ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.278563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1410db1-1f6f-43eb-a670-3f03788329ee None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.278855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1410db1-1f6f-43eb-a670-3f03788329ee None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.279138 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1410db1-1f6f-43eb-a670-3f03788329ee None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.279969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1410db1-1f6f-43eb-a670-3f03788329ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:51.280354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1410db1-1f6f-43eb-a670-3f03788329ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:51.281759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1410db1-1f6f-43eb-a670-3f03788329ee None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:51.282520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 36/72] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:51.283735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.295406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 37/73] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:51.296950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c8d7e-667a-4d4c-8e7b-a5c7a938132a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.297403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c8d7e-667a-4d4c-8e7b-a5c7a938132a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.297842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c8d7e-667a-4d4c-8e7b-a5c7a938132a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.314549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-025c8d7e-667a-4d4c-8e7b-a5c7a938132a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:51.316081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-025c8d7e-667a-4d4c-8e7b-a5c7a938132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:51.382662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 38/74] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:51.383545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c8d7e-667a-4d4c-8e7b-a5c7a938132a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.391604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.398075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae077a-5d47-4996-83fa-3ae160f30e96 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.431471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.432149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.432436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.432714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.433477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:51.433792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:51.435240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:51.443842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 37/75] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:51.444629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.488940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c8d7e-667a-4d4c-8e7b-a5c7a938132a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.504267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.525512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479cdf1-9ca6-4b6b-af50-5f86d68099cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.526414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479cdf1-9ca6-4b6b-af50-5f86d68099cb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.526695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479cdf1-9ca6-4b6b-af50-5f86d68099cb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.526971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479cdf1-9ca6-4b6b-af50-5f86d68099cb None admin] PATH_INFO: `/v3/users/76b6ddf40257432aa78fadccd4348cd5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.528227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479cdf1-9ca6-4b6b-af50-5f86d68099cb None admin] RBAC: Authorizing `identity:get_user(user_id=76b6ddf40257432aa78fadccd4348cd5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:51.529686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479cdf1-9ca6-4b6b-af50-5f86d68099cb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:51.531688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 39/76] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/users/76b6ddf40257432aa78fadccd4348cd5 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:51.532862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc68010-848a-4f1d-a5a0-779c056cf7db None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.533212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc68010-848a-4f1d-a5a0-779c056cf7db None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.533388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc68010-848a-4f1d-a5a0-779c056cf7db None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.534802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 40/77] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:51.536966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc68010-848a-4f1d-a5a0-779c056cf7db None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.552265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7626b8-e172-49a5-aecd-a8b8f2bfd586 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.585923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07a72c-6a9f-4b6f-b3b2-087a3c487f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.586808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a72c-6a9f-4b6f-b3b2-087a3c487f7b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.587139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a72c-6a9f-4b6f-b3b2-087a3c487f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.587774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a72c-6a9f-4b6f-b3b2-087a3c487f7b None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.601161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a72c-6a9f-4b6f-b3b2-087a3c487f7b None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:51.602661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a72c-6a9f-4b6f-b3b2-087a3c487f7b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:51.604390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 38/78] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:51.605156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07a72c-6a9f-4b6f-b3b2-087a3c487f7b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.612782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.638969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc68010-848a-4f1d-a5a0-779c056cf7db None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.672031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba47fd5-7222-4fc4-b868-334101a35524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.672846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba47fd5-7222-4fc4-b868-334101a35524 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.673132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba47fd5-7222-4fc4-b868-334101a35524 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.673409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba47fd5-7222-4fc4-b868-334101a35524 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.683564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba47fd5-7222-4fc4-b868-334101a35524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9ba47fd5-7222-4fc4-b868-334101a35524 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:51.691327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:51.692247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:51.693020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 41/79] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:51.694101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba47fd5-7222-4fc4-b868-334101a35524 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.710404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07a72c-6a9f-4b6f-b3b2-087a3c487f7b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.726972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.745609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15c6ea-106e-421d-9058-78ea75e0de7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.746500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15c6ea-106e-421d-9058-78ea75e0de7e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.747087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15c6ea-106e-421d-9058-78ea75e0de7e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.747419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15c6ea-106e-421d-9058-78ea75e0de7e None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.748620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15c6ea-106e-421d-9058-78ea75e0de7e None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:51.750099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15c6ea-106e-421d-9058-78ea75e0de7e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:51.751768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 39/80] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:51.753006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3d545-1e38-4ee8-9f47-a056b35c03ce None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.753305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3d545-1e38-4ee8-9f47-a056b35c03ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.753576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3d545-1e38-4ee8-9f47-a056b35c03ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.770149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f0c3d545-1e38-4ee8-9f47-a056b35c03ce None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:51.771384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f0c3d545-1e38-4ee8-9f47-a056b35c03ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:51.795383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba47fd5-7222-4fc4-b868-334101a35524 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.829408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425302ee-64ce-4a2d-aec8-4919ca6587f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.830059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425302ee-64ce-4a2d-aec8-4919ca6587f7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.830332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425302ee-64ce-4a2d-aec8-4919ca6587f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.830612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425302ee-64ce-4a2d-aec8-4919ca6587f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:51.833636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.838669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 40/81] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:51.839538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3d545-1e38-4ee8-9f47-a056b35c03ce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:51.905116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425302ee-64ce-4a2d-aec8-4919ca6587f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:51.906538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425302ee-64ce-4a2d-aec8-4919ca6587f7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:51.945630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3d545-1e38-4ee8-9f47-a056b35c03ce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:51.959928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:51.980124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494c27-23dd-4761-a3c1-9e51f62488a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:51.980776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494c27-23dd-4761-a3c1-9e51f62488a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:51.981061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494c27-23dd-4761-a3c1-9e51f62488a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:51.981334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494c27-23dd-4761-a3c1-9e51f62488a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.004022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 42/82] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:06:52.007153 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425302ee-64ce-4a2d-aec8-4919ca6587f7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.041617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:52.052597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94494c27-23dd-4761-a3c1-9e51f62488a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.054293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94494c27-23dd-4761-a3c1-9e51f62488a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.106167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425302ee-64ce-4a2d-aec8-4919ca6587f7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.140019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.140671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.140958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.141226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.141950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:52.142253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.143672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.151053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 41/83] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:06:52.156363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 43/84] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:52.156494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494c27-23dd-4761-a3c1-9e51f62488a8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.156581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.170323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:52.249424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:52.255267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494c27-23dd-4761-a3c1-9e51f62488a8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.256536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331939b-4356-4179-b196-d39fd9d8df29 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.290064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4627cb1-f0b6-43bb-89e3-58367252dbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.290865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4627cb1-f0b6-43bb-89e3-58367252dbcc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.290963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac79b2b-97fd-4760-a489-d3a12b3d882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.291150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4627cb1-f0b6-43bb-89e3-58367252dbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.291454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4627cb1-f0b6-43bb-89e3-58367252dbcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.291644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac79b2b-97fd-4760-a489-d3a12b3d882f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.291937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac79b2b-97fd-4760-a489-d3a12b3d882f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.292221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac79b2b-97fd-4760-a489-d3a12b3d882f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.292635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4627cb1-f0b6-43bb-89e3-58367252dbcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.294171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4627cb1-f0b6-43bb-89e3-58367252dbcc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.295840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 44/85] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:52.296624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4627cb1-f0b6-43bb-89e3-58367252dbcc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.361077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac79b2b-97fd-4760-a489-d3a12b3d882f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.362579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac79b2b-97fd-4760-a489-d3a12b3d882f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.377177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:52.396821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4627cb1-f0b6-43bb-89e3-58367252dbcc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.432432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e9e6-a8ba-4fe0-86d1-93000f28febb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.433229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e9e6-a8ba-4fe0-86d1-93000f28febb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.433516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e9e6-a8ba-4fe0-86d1-93000f28febb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.433796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e9e6-a8ba-4fe0-86d1-93000f28febb None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.435032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77e9e6-a8ba-4fe0-86d1-93000f28febb None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.436532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77e9e6-a8ba-4fe0-86d1-93000f28febb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.438400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 45/86] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:52.438983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e9e6-a8ba-4fe0-86d1-93000f28febb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.466116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 42/87] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:06:52.466939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac79b2b-97fd-4760-a489-d3a12b3d882f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.473782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:52.548219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e9e6-a8ba-4fe0-86d1-93000f28febb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.568480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac79b2b-97fd-4760-a489-d3a12b3d882f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.581898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c703-366e-4309-96c2-fd830996580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.582552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:52.582779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c703-366e-4309-96c2-fd830996580b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.583068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c703-366e-4309-96c2-fd830996580b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.583378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c703-366e-4309-96c2-fd830996580b None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.584629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736c703-366e-4309-96c2-fd830996580b None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.586318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736c703-366e-4309-96c2-fd830996580b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.588078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 46/88] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:52.588835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c703-366e-4309-96c2-fd830996580b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.609756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b16a0-8e08-4f43-9c5e-277df6826e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.610650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b16a0-8e08-4f43-9c5e-277df6826e7a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.610921 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b16a0-8e08-4f43-9c5e-277df6826e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.611230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b16a0-8e08-4f43-9c5e-277df6826e7a None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.612459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b16a0-8e08-4f43-9c5e-277df6826e7a None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.613940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b16a0-8e08-4f43-9c5e-277df6826e7a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.615936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 43/89] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:52.616526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b16a0-8e08-4f43-9c5e-277df6826e7a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.691086 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:52.694108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c703-366e-4309-96c2-fd830996580b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.728904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286afa7-0adf-4c6d-83fb-6d81ae3e5d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.729574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286afa7-0adf-4c6d-83fb-6d81ae3e5d12 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.729873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286afa7-0adf-4c6d-83fb-6d81ae3e5d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.730136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286afa7-0adf-4c6d-83fb-6d81ae3e5d12 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.730734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b16a0-8e08-4f43-9c5e-277df6826e7a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.730869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286afa7-0adf-4c6d-83fb-6d81ae3e5d12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.732358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286afa7-0adf-4c6d-83fb-6d81ae3e5d12 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.759744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 47/90] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:06:52 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:06:52.760566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286afa7-0adf-4c6d-83fb-6d81ae3e5d12 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.765165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.765846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.766131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.766407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.767288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=20862a0ad40a41af9af9f940f3d4c544, user.id=76b6ddf40257432aa78fadccd4348cd5) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:52.767607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.769108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.775795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 44/91] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/role_assignments?user.id=76b6ddf40257432aa78fadccd4348cd5&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:52.777167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.849566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:52.910396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286afa7-0adf-4c6d-83fb-6d81ae3e5d12 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.920591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cb770-987a-4054-9a57-01ffdc0d4de7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:52.956013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272d43-2639-4a0f-baef-ba0a3179a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.956837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272d43-2639-4a0f-baef-ba0a3179a4ea None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.957102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272d43-2639-4a0f-baef-ba0a3179a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.957380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272d43-2639-4a0f-baef-ba0a3179a4ea None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.958807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5272d43-2639-4a0f-baef-ba0a3179a4ea None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.960634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5272d43-2639-4a0f-baef-ba0a3179a4ea None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.962944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 48/92] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:52.963261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272d43-2639-4a0f-baef-ba0a3179a4ea None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:52.965589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:52.966219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:52.966503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:52.966778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:52.967719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] RBAC: Adding query filter params (role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=20862a0ad40a41af9af9f940f3d4c544, user.id=9f5f4a174ee24efbbdabdbbd9715f21e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:52.968032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:52.970419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:52.983253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 45/93] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/role_assignments?user.id=9f5f4a174ee24efbbdabdbbd9715f21e&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=798b7bcb53684ab49ae899f8902f444a => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:52.983253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:53.019667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:53.108784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3299f6b-d5d6-49ea-8e90-1fae18b87b71 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:53.113433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272d43-2639-4a0f-baef-ba0a3179a4ea None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:53.126613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:53.162252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:53.163380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:53.163839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:53.164294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:53.165152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] RBAC: Adding query filter params (role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=20862a0ad40a41af9af9f940f3d4c544, user.id=9f5f4a174ee24efbbdabdbbd9715f21e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:53.165467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:53.166816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b6ee9-91ce-4f6e-9984-3df4ce146cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:53.166942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:53.167844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b6ee9-91ce-4f6e-9984-3df4ce146cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:53.168353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b6ee9-91ce-4f6e-9984-3df4ce146cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:53.168825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b6ee9-91ce-4f6e-9984-3df4ce146cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:53.178881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 46/94] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/role_assignments?user.id=9f5f4a174ee24efbbdabdbbd9715f21e&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:53.255944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b6ee9-91ce-4f6e-9984-3df4ce146cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:53.258270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b6ee9-91ce-4f6e-9984-3df4ce146cb7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:53.288352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:53.373697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 49/95] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:06:53.379534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:53.498471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4281db7-1613-4214-b911-4da5f4129d79 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:53.539962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbc0ac-86b8-404f-be99-07a1792fc33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:53.540849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc0ac-86b8-404f-be99-07a1792fc33a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:53.541130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc0ac-86b8-404f-be99-07a1792fc33a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:53.541406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc0ac-86b8-404f-be99-07a1792fc33a None admin] PATH_INFO: `/v3/projects/6aa34392639c42f1a4cb470639efb105` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:53.543045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbc0ac-86b8-404f-be99-07a1792fc33a None admin] RBAC: Authorizing `identity:get_project(project_id=6aa34392639c42f1a4cb470639efb105)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:53.544743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbc0ac-86b8-404f-be99-07a1792fc33a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:53.546411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 47/96] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:53 2022] GET /identity/v3/projects/6aa34392639c42f1a4cb470639efb105 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:53.553444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b6ee9-91ce-4f6e-9984-3df4ce146cb7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:53.670567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b6ee9-91ce-4f6e-9984-3df4ce146cb7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:53.694872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:53.709809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88faef-ab27-4401-bf82-0a89b3944560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:53.710473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88faef-ab27-4401-bf82-0a89b3944560 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:53.710751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88faef-ab27-4401-bf82-0a89b3944560 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:53.711026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88faef-ab27-4401-bf82-0a89b3944560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:53.711890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88faef-ab27-4401-bf82-0a89b3944560 None admin] RBAC: Adding query filter params (role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=6aa34392639c42f1a4cb470639efb105, user.id=9f5f4a174ee24efbbdabdbbd9715f21e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:53.712201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88faef-ab27-4401-bf82-0a89b3944560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:53.713663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88faef-ab27-4401-bf82-0a89b3944560 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:53.719869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 50/97] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:06:53 2022] GET /identity/v3/role_assignments?user.id=9f5f4a174ee24efbbdabdbbd9715f21e&scope.project.id=6aa34392639c42f1a4cb470639efb105&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:55.578052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9525c-2fa1-4f76-b0b3-f4699841ed06 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:55.578600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9525c-2fa1-4f76-b0b3-f4699841ed06 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:55.578600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9525c-2fa1-4f76-b0b3-f4699841ed06 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:55.580010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 48/98] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:55.595891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23ca43-b149-4872-8ee6-634b4f9e84e7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:55.596362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23ca43-b149-4872-8ee6-634b4f9e84e7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:55.596462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23ca43-b149-4872-8ee6-634b4f9e84e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:55.599412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4e54-0f5a-47a4-a07f-a60575136636 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:55.600000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4e54-0f5a-47a4-a07f-a60575136636 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:55.600577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4e54-0f5a-47a4-a07f-a60575136636 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:55.602156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 49/99] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:55.607500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8534b-b40d-4564-94d0-277134e04754 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:55.608024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8534b-b40d-4564-94d0-277134e04754 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:55.608523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8534b-b40d-4564-94d0-277134e04754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:55.620222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2d23ca43-b149-4872-8ee6-634b4f9e84e7 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:55.621426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2d23ca43-b149-4872-8ee6-634b4f9e84e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:55.626552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7fd8534b-b40d-4564-94d0-277134e04754 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:55.628002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7fd8534b-b40d-4564-94d0-277134e04754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:55.666349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:55.701004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 50/100] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:55.703026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a982b-093f-4b18-94c4-f29072567651 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:55.703485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a982b-093f-4b18-94c4-f29072567651 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:55.703932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a982b-093f-4b18-94c4-f29072567651 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:55.705357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 51/101] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:55.716178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03ac61-15af-4547-b850-6411da057707 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:55.716660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03ac61-15af-4547-b850-6411da057707 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:55.717084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03ac61-15af-4547-b850-6411da057707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:55.737420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9c03ac61-15af-4547-b850-6411da057707 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:55.738831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9c03ac61-15af-4547-b850-6411da057707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:55.805235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 51/102] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:06:55.806021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23ca43-b149-4872-8ee6-634b4f9e84e7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:55.812337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 52/103] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:55.849818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03ac61-15af-4547-b850-6411da057707 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:55.978894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23ca43-b149-4872-8ee6-634b4f9e84e7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.015616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:56.048037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ecda0-584f-446e-8024-0716891dbf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.048867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ecda0-584f-446e-8024-0716891dbf55 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.049148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ecda0-584f-446e-8024-0716891dbf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.049426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ecda0-584f-446e-8024-0716891dbf55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.055967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03ac61-15af-4547-b850-6411da057707 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.057493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:56.072310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ecda0-584f-446e-8024-0716891dbf55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e19ecda0-584f-446e-8024-0716891dbf55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:56.080513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:56.081560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:06:56.082518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:56.088374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 52/104] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:06:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:56.095864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ecda0-584f-446e-8024-0716891dbf55 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.129394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed885-48f9-4dea-9dc3-ed765c7ce64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.130378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed885-48f9-4dea-9dc3-ed765c7ce64a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.130761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed885-48f9-4dea-9dc3-ed765c7ce64a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.131254 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed885-48f9-4dea-9dc3-ed765c7ce64a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.142763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fed885-48f9-4dea-9dc3-ed765c7ce64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-97fed885-48f9-4dea-9dc3-ed765c7ce64a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:56.150916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:56.151853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:06:56.152640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:06:56.155133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 53/105] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:56.156145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed885-48f9-4dea-9dc3-ed765c7ce64a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.225104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ecda0-584f-446e-8024-0716891dbf55 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.262998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ccc3a-2e4a-479f-8f7f-7697a5d84ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.264568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ccc3a-2e4a-479f-8f7f-7697a5d84ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.265063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ccc3a-2e4a-479f-8f7f-7697a5d84ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.265672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ccc3a-2e4a-479f-8f7f-7697a5d84ba0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.275715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed885-48f9-4dea-9dc3-ed765c7ce64a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.280375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ccc3a-2e4a-479f-8f7f-7697a5d84ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-648ccc3a-2e4a-479f-8f7f-7697a5d84ba0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:56.290230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:56.291079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:56.292083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 53/106] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:56.293810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc38f4-0501-4d11-846e-49c6f0413dbe None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.294180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc38f4-0501-4d11-846e-49c6f0413dbe None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.294530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc38f4-0501-4d11-846e-49c6f0413dbe None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.296164 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 54/107] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:56.299838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc38f4-0501-4d11-846e-49c6f0413dbe None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.320234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfc1af-f846-4a21-89c3-eea7a6a202c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.321417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfc1af-f846-4a21-89c3-eea7a6a202c3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.321896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfc1af-f846-4a21-89c3-eea7a6a202c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.322357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfc1af-f846-4a21-89c3-eea7a6a202c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.323349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfc1af-f846-4a21-89c3-eea7a6a202c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:56.323877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfc1af-f846-4a21-89c3-eea7a6a202c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:56.325672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfc1af-f846-4a21-89c3-eea7a6a202c3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:56.335620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 54/108] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:56.341133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32309de7-a929-4ca2-b08b-4c39ed13069a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.341638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32309de7-a929-4ca2-b08b-4c39ed13069a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.342105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32309de7-a929-4ca2-b08b-4c39ed13069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.355146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:56.364066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32309de7-a929-4ca2-b08b-4c39ed13069a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:56.365571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32309de7-a929-4ca2-b08b-4c39ed13069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:56.385882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:56.427741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc38f4-0501-4d11-846e-49c6f0413dbe None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.460475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 55/109] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:56.461548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32309de7-a929-4ca2-b08b-4c39ed13069a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.468008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.468975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.469495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.470032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.470977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:56.471559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:56.473341 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:56.482555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 55/110] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:56.483854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.578176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32309de7-a929-4ca2-b08b-4c39ed13069a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.589595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:56.592892 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b360918-9be4-4250-bdfc-e075ccf728e0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.616132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.616879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.617292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.617575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.618334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:56.618631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:56.620195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:56.628464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a8b79-124f-47f7-be2c-d355a497d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.629354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8b79-124f-47f7-be2c-d355a497d523 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.629517 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 56/111] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:56.629937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8b79-124f-47f7-be2c-d355a497d523 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.630222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8b79-124f-47f7-be2c-d355a497d523 None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.630533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.631636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a8b79-124f-47f7-be2c-d355a497d523 None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:56.633360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a8b79-124f-47f7-be2c-d355a497d523 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:56.639624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 56/112] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:56.639624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad0a6f-d61a-4c9f-9eda-90b74abb5070 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.639624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad0a6f-d61a-4c9f-9eda-90b74abb5070 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.639624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad0a6f-d61a-4c9f-9eda-90b74abb5070 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.639624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 57/113] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:56.640122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad0a6f-d61a-4c9f-9eda-90b74abb5070 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.657258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:56.746273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73c712-5964-4b0f-831a-bbd75a08f2d5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.749036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad0a6f-d61a-4c9f-9eda-90b74abb5070 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.782533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8b4d1-5726-45dd-8829-6980edb1a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.783468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8b4d1-5726-45dd-8829-6980edb1a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.783723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8b4d1-5726-45dd-8829-6980edb1a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.783999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8b4d1-5726-45dd-8829-6980edb1a3a5 None admin] PATH_INFO: `/v3/users/76b6ddf40257432aa78fadccd4348cd5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.784111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0bcd6-0fd4-4db0-a129-79bb233298b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.784866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0bcd6-0fd4-4db0-a129-79bb233298b5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.785150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0bcd6-0fd4-4db0-a129-79bb233298b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.785241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8b4d1-5726-45dd-8829-6980edb1a3a5 None admin] RBAC: Authorizing `identity:get_user(user_id=76b6ddf40257432aa78fadccd4348cd5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:56.785428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0bcd6-0fd4-4db0-a129-79bb233298b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.786674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8b4d1-5726-45dd-8829-6980edb1a3a5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:56.788619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 57/114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/users/76b6ddf40257432aa78fadccd4348cd5 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:56.789173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8b4d1-5726-45dd-8829-6980edb1a3a5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.800733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0bcd6-0fd4-4db0-a129-79bb233298b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-08c0bcd6-0fd4-4db0-a129-79bb233298b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:56.808846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:56.809734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:06:56.810527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:56.811673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 58/115] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:56.811673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0bcd6-0fd4-4db0-a129-79bb233298b5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.844964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:56.886585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:56.902260 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8b4d1-5726-45dd-8829-6980edb1a3a5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.921971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0bcd6-0fd4-4db0-a129-79bb233298b5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:56.935090 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc2223-8eef-403e-bcbb-b4c8b57b1703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.936445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2223-8eef-403e-bcbb-b4c8b57b1703 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.936761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2223-8eef-403e-bcbb-b4c8b57b1703 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.937037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2223-8eef-403e-bcbb-b4c8b57b1703 None admin] PATH_INFO: `/v3/users/a8fa1faa859d4ccdac4c6ad255b8cf42` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.950389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc2223-8eef-403e-bcbb-b4c8b57b1703 None admin] RBAC: Authorizing `identity:get_user(user_id=a8fa1faa859d4ccdac4c6ad255b8cf42)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:56.951967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc2223-8eef-403e-bcbb-b4c8b57b1703 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:56.953854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 58/116] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/users/a8fa1faa859d4ccdac4c6ad255b8cf42 => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:56.954944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c9513-578e-4773-9077-f385eaa666f2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.955147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c9513-578e-4773-9077-f385eaa666f2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.955502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c9513-578e-4773-9077-f385eaa666f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.956133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee924c-9274-45b5-99e1-3b0860be5007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:56.957080 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee924c-9274-45b5-99e1-3b0860be5007 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:56.957325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee924c-9274-45b5-99e1-3b0860be5007 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:56.957719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee924c-9274-45b5-99e1-3b0860be5007 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:56.959045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee924c-9274-45b5-99e1-3b0860be5007 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:56.960574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee924c-9274-45b5-99e1-3b0860be5007 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:56.962426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 59/117] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:56.963240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee924c-9274-45b5-99e1-3b0860be5007 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:56.973255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-093c9513-578e-4773-9077-f385eaa666f2 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:56.974484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-093c9513-578e-4773-9077-f385eaa666f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:57.041399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 59/118] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:57.042230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c9513-578e-4773-9077-f385eaa666f2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.057328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:57.064975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee924c-9274-45b5-99e1-3b0860be5007 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.098395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de196b9-e47c-4126-8b93-20210d739e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.099191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de196b9-e47c-4126-8b93-20210d739e33 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.099498 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de196b9-e47c-4126-8b93-20210d739e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.099810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de196b9-e47c-4126-8b93-20210d739e33 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.101157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de196b9-e47c-4126-8b93-20210d739e33 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.102630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de196b9-e47c-4126-8b93-20210d739e33 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.104360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 60/119] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:56 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:57.105582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773bf24-3ec3-4913-bde1-28d081aab8ac None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.105928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773bf24-3ec3-4913-bde1-28d081aab8ac None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.106214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773bf24-3ec3-4913-bde1-28d081aab8ac None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.107480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 61/120] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:06:57.108161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773bf24-3ec3-4913-bde1-28d081aab8ac None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.126513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:57.155923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c9513-578e-4773-9077-f385eaa666f2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.190513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.191199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.191509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.191785 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.192542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:57.192851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.194317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.202723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 60/121] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:06:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:57.203523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.214971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773bf24-3ec3-4913-bde1-28d081aab8ac None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.249650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650e1d-9c74-4a3e-83ac-3f1d1d01449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.250301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650e1d-9c74-4a3e-83ac-3f1d1d01449d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.250587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650e1d-9c74-4a3e-83ac-3f1d1d01449d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.250854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650e1d-9c74-4a3e-83ac-3f1d1d01449d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.281679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:57.299816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc046a-c15c-4693-9817-fe483baa2c6c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.318457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4650e1d-9c74-4a3e-83ac-3f1d1d01449d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.320103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4650e1d-9c74-4a3e-83ac-3f1d1d01449d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.337121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ef5e1-ec1b-4452-a0b8-2c2313bc70a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.338027 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ef5e1-ec1b-4452-a0b8-2c2313bc70a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.338302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ef5e1-ec1b-4452-a0b8-2c2313bc70a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.338571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ef5e1-ec1b-4452-a0b8-2c2313bc70a1 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544/users/9f5f4a174ee24efbbdabdbbd9715f21e/roles/798b7bcb53684ab49ae899f8902f444a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.340816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ef5e1-ec1b-4452-a0b8-2c2313bc70a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=20862a0ad40a41af9af9f940f3d4c544, user_id=9f5f4a174ee24efbbdabdbbd9715f21e, role_id=798b7bcb53684ab49ae899f8902f444a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.342284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ef5e1-ec1b-4452-a0b8-2c2313bc70a1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.344549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:57.357629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 61/122] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:06:57 2022] PUT /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544/users/9f5f4a174ee24efbbdabdbbd9715f21e/roles/798b7bcb53684ab49ae899f8902f444a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:06:57.358441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ef5e1-ec1b-4452-a0b8-2c2313bc70a1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.435160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 62/123] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:06:57.436054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650e1d-9c74-4a3e-83ac-3f1d1d01449d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.480886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ef5e1-ec1b-4452-a0b8-2c2313bc70a1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.518688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:57.521099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce90d2-2d10-4407-a6e6-5cbd5150efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.522388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce90d2-2d10-4407-a6e6-5cbd5150efb2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.522655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce90d2-2d10-4407-a6e6-5cbd5150efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.522926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce90d2-2d10-4407-a6e6-5cbd5150efb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.524160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce90d2-2d10-4407-a6e6-5cbd5150efb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.525648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce90d2-2d10-4407-a6e6-5cbd5150efb2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.527455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 62/124] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:06:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:57.528649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32160107-aa3b-4e8d-9c13-43cbed5aba49 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.529026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32160107-aa3b-4e8d-9c13-43cbed5aba49 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.529326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32160107-aa3b-4e8d-9c13-43cbed5aba49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.545534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32160107-aa3b-4e8d-9c13-43cbed5aba49 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:06:57.546713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32160107-aa3b-4e8d-9c13-43cbed5aba49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:06:57.561705 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650e1d-9c74-4a3e-83ac-3f1d1d01449d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.598151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cba89-8526-4742-86e0-d28502dd361d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.599404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cba89-8526-4742-86e0-d28502dd361d None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.600087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cba89-8526-4742-86e0-d28502dd361d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.600609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cba89-8526-4742-86e0-d28502dd361d None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544/users/76b6ddf40257432aa78fadccd4348cd5/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.603164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:57.603744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cba89-8526-4742-86e0-d28502dd361d None admin] RBAC: Authorizing `identity:create_grant(project_id=20862a0ad40a41af9af9f940f3d4c544, user_id=76b6ddf40257432aa78fadccd4348cd5, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.606106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cba89-8526-4742-86e0-d28502dd361d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.614965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 63/125] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:06:57.615341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32160107-aa3b-4e8d-9c13-43cbed5aba49 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.639233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 63/126] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:06:57 2022] PUT /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544/users/76b6ddf40257432aa78fadccd4348cd5/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:06:57.639233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cba89-8526-4742-86e0-d28502dd361d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.770267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32160107-aa3b-4e8d-9c13-43cbed5aba49 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.771903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cba89-8526-4742-86e0-d28502dd361d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.798391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:57.810464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b876ad-2543-4add-9ec2-3949e2d487ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.810794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60e355-869f-4753-8134-95d55253641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.811737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60e355-869f-4753-8134-95d55253641b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.811860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b876ad-2543-4add-9ec2-3949e2d487ba None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.812013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60e355-869f-4753-8134-95d55253641b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.812286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60e355-869f-4753-8134-95d55253641b None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.812381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b876ad-2543-4add-9ec2-3949e2d487ba None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.812749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b876ad-2543-4add-9ec2-3949e2d487ba None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.813616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60e355-869f-4753-8134-95d55253641b None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.814242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b876ad-2543-4add-9ec2-3949e2d487ba None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.815107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60e355-869f-4753-8134-95d55253641b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.815930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b876ad-2543-4add-9ec2-3949e2d487ba None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.816769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 64/127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:57 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:57.817586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60e355-869f-4753-8134-95d55253641b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.818559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 64/128] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:57 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:57.819437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b876ad-2543-4add-9ec2-3949e2d487ba None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.884413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:57.927494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60e355-869f-4753-8134-95d55253641b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.928210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b876ad-2543-4add-9ec2-3949e2d487ba None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:57.962403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a7b4-301d-49eb-be63-5d6c7d9b932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.962844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ca84a-4802-4668-9cac-2e4d5d27ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:57.963551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ca84a-4802-4668-9cac-2e4d5d27ac48 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.963825 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ca84a-4802-4668-9cac-2e4d5d27ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.964100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ca84a-4802-4668-9cac-2e4d5d27ac48 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.964812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ca84a-4802-4668-9cac-2e4d5d27ac48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:57.965024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a7b4-301d-49eb-be63-5d6c7d9b932e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:57.965527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a7b4-301d-49eb-be63-5d6c7d9b932e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:57.965939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a7b4-301d-49eb-be63-5d6c7d9b932e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:57.966278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ca84a-4802-4668-9cac-2e4d5d27ac48 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:57.978277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521a7b4-301d-49eb-be63-5d6c7d9b932e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2521a7b4-301d-49eb-be63-5d6c7d9b932e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:06:57.987241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:06:57.988407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 65/129] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:06:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:06:57.989345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a7b4-301d-49eb-be63-5d6c7d9b932e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:57.999952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 65/130] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:06:57 2022] POST /identity/v3/projects => generated 302 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:06:58.001255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ca84a-4802-4668-9cac-2e4d5d27ac48 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:58.047891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:58.098481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a7b4-301d-49eb-be63-5d6c7d9b932e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:58.117539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ca84a-4802-4668-9cac-2e4d5d27ac48 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:58.135229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9409-85bc-442c-a83c-6215fdfb10c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:58.135603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:58.136058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9409-85bc-442c-a83c-6215fdfb10c0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:58.136414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9409-85bc-442c-a83c-6215fdfb10c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:58.136622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9409-85bc-442c-a83c-6215fdfb10c0 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:58.138079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc9409-85bc-442c-a83c-6215fdfb10c0 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:58.139828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc9409-85bc-442c-a83c-6215fdfb10c0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:58.141999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 66/131] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:57 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:58.142673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9409-85bc-442c-a83c-6215fdfb10c0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:58.162666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:58.163634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:58.164078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:58.164515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:58.165513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] RBAC: Adding query filter params (role.id=e7d8e4fc6f35425ab4e5c87ed577de1f, scope.project.id=20862a0ad40a41af9af9f940f3d4c544, user.id=a8fa1faa859d4ccdac4c6ad255b8cf42) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:58.165983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:58.167659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:58.174342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 66/132] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:06:58 2022] GET /identity/v3/role_assignments?user.id=a8fa1faa859d4ccdac4c6ad255b8cf42&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=e7d8e4fc6f35425ab4e5c87ed577de1f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:58.175576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:58.260223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9409-85bc-442c-a83c-6215fdfb10c0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:58.287919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:58.302092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c472b6-4143-4b41-b0ca-b162a67580f8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:58.305226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:58.306241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:58.306672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:58.307097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:58.308354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:06:58.308836 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:58.310387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:58.323862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 67/133] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:06:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:58.329974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:58.340722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98792910-53b8-4280-a854-5603fa2733b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:58.341638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98792910-53b8-4280-a854-5603fa2733b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:58.341916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98792910-53b8-4280-a854-5603fa2733b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:58.342190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98792910-53b8-4280-a854-5603fa2733b2 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544/users/9f5f4a174ee24efbbdabdbbd9715f21e/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:58.344217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98792910-53b8-4280-a854-5603fa2733b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=20862a0ad40a41af9af9f940f3d4c544, user_id=9f5f4a174ee24efbbdabdbbd9715f21e, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:58.345714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98792910-53b8-4280-a854-5603fa2733b2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:58.368178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 67/134] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:06:58 2022] PUT /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544/users/9f5f4a174ee24efbbdabdbbd9715f21e/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:06:58.476477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661cfbe-4341-4e15-bf6a-242ad5f04cbf None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:58.520124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27241173-84d3-4e66-a417-55bd79b9a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:58.521182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27241173-84d3-4e66-a417-55bd79b9a882 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:58.521559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27241173-84d3-4e66-a417-55bd79b9a882 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:58.521998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27241173-84d3-4e66-a417-55bd79b9a882 None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:58.523428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27241173-84d3-4e66-a417-55bd79b9a882 None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:58.525120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27241173-84d3-4e66-a417-55bd79b9a882 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:58.527718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 68/135] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:06:58 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:58.533106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98792910-53b8-4280-a854-5603fa2733b2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:58.608505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:58.689001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98792910-53b8-4280-a854-5603fa2733b2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:58.729693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3f83a-8ef4-42ea-8205-d9e53406cec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:58.731020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f83a-8ef4-42ea-8205-d9e53406cec8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:58.731665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f83a-8ef4-42ea-8205-d9e53406cec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:58.731952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f83a-8ef4-42ea-8205-d9e53406cec8 None admin] PATH_INFO: `/v3/projects/6aa34392639c42f1a4cb470639efb105` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:58.733588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3f83a-8ef4-42ea-8205-d9e53406cec8 None admin] RBAC: Authorizing `identity:get_project(project_id=6aa34392639c42f1a4cb470639efb105)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:58.735830 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3f83a-8ef4-42ea-8205-d9e53406cec8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:58.737560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 68/136] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:06:58 2022] GET /identity/v3/projects/6aa34392639c42f1a4cb470639efb105 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:06:58.743584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27241173-84d3-4e66-a417-55bd79b9a882 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:06:58.803773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:06:58.857044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27241173-84d3-4e66-a417-55bd79b9a882 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:06:58.894415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2cadb-0173-41cc-9e00-60b6f604c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:06:58.895358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2cadb-0173-41cc-9e00-60b6f604c063 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:06:58.895642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2cadb-0173-41cc-9e00-60b6f604c063 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:06:58.895918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2cadb-0173-41cc-9e00-60b6f604c063 None admin] PATH_INFO: `/v3/projects/6aa34392639c42f1a4cb470639efb105/users/9f5f4a174ee24efbbdabdbbd9715f21e/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:06:58.900540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2cadb-0173-41cc-9e00-60b6f604c063 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aa34392639c42f1a4cb470639efb105, user_id=9f5f4a174ee24efbbdabdbbd9715f21e, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:06:58.902003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2cadb-0173-41cc-9e00-60b6f604c063 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:06:58.924430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 69/137] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:06:58 2022] PUT /identity/v3/projects/6aa34392639c42f1a4cb470639efb105/users/9f5f4a174ee24efbbdabdbbd9715f21e/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:00.290754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d81c12-1b3e-4ae1-930c-c2ad74e3c898 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:00.291514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d81c12-1b3e-4ae1-930c-c2ad74e3c898 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:00.291514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d81c12-1b3e-4ae1-930c-c2ad74e3c898 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:00.292752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 69/138] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:00.298094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eedf76-867f-4904-b502-5891c74823ef None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:00.298596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eedf76-867f-4904-b502-5891c74823ef None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:00.298768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eedf76-867f-4904-b502-5891c74823ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:00.315544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-07eedf76-867f-4904-b502-5891c74823ef None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:00.316732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-07eedf76-867f-4904-b502-5891c74823ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:00.384414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 70/139] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:00.393417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda48ed-c8a8-4fc4-85fc-d86b02fe0391 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:00.393709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda48ed-c8a8-4fc4-85fc-d86b02fe0391 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:00.393994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda48ed-c8a8-4fc4-85fc-d86b02fe0391 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:00.395396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 70/140] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:00.400921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7286a-92f8-4b21-b8ca-1c6eda4a4720 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:00.401218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7286a-92f8-4b21-b8ca-1c6eda4a4720 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:00.401504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7286a-92f8-4b21-b8ca-1c6eda4a4720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:00.421443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d7e7286a-92f8-4b21-b8ca-1c6eda4a4720 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:00.422642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d7e7286a-92f8-4b21-b8ca-1c6eda4a4720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:00.443166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda48ed-c8a8-4fc4-85fc-d86b02fe0391 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:00.511702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:00.591126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 71/141] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:00.609202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda48ed-c8a8-4fc4-85fc-d86b02fe0391 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:00.612172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:00.630871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7286a-92f8-4b21-b8ca-1c6eda4a4720 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:00.663151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf7c6d-889c-4843-b5be-47eb4d09ce9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:00.663998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf7c6d-889c-4843-b5be-47eb4d09ce9d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:00.664282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf7c6d-889c-4843-b5be-47eb4d09ce9d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:00.664557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf7c6d-889c-4843-b5be-47eb4d09ce9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:00.682218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf7c6d-889c-4843-b5be-47eb4d09ce9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-11cf7c6d-889c-4843-b5be-47eb4d09ce9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:00.690658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:00.693292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:00.695269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 71/142] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:00.699240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf7c6d-889c-4843-b5be-47eb4d09ce9d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:00.834622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf7c6d-889c-4843-b5be-47eb4d09ce9d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:00.868139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:00.868822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:00.869103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:00.869384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:00.870132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:00.870439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:00.872232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:00.880913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 72/143] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:00.888181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:00.893185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7286a-92f8-4b21-b8ca-1c6eda4a4720 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:00.938119 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:00.957108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53340037-96f3-42f6-9d1e-e973140477c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:00.957935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53340037-96f3-42f6-9d1e-e973140477c2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:00.958215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53340037-96f3-42f6-9d1e-e973140477c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:00.958488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53340037-96f3-42f6-9d1e-e973140477c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:00.976721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53340037-96f3-42f6-9d1e-e973140477c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-53340037-96f3-42f6-9d1e-e973140477c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:00.984846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:00.985891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 72/144] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:00.988057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58317fa9-48ac-4bfd-9e79-c9d18949242e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:00.989529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58317fa9-48ac-4bfd-9e79-c9d18949242e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:00.989529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58317fa9-48ac-4bfd-9e79-c9d18949242e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:00.989529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 73/145] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:00.995231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58317fa9-48ac-4bfd-9e79-c9d18949242e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:01.024260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:01.090417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3593b-3fad-47f7-8061-ca8d0d125d7f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:01.154317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58317fa9-48ac-4bfd-9e79-c9d18949242e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:01.158554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf027ce9-79d5-404d-93b8-7f1ef1fbfd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:01.159478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf027ce9-79d5-404d-93b8-7f1ef1fbfd1e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.159763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf027ce9-79d5-404d-93b8-7f1ef1fbfd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.160295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf027ce9-79d5-404d-93b8-7f1ef1fbfd1e None admin] PATH_INFO: `/v3/users/76b6ddf40257432aa78fadccd4348cd5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.166249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf027ce9-79d5-404d-93b8-7f1ef1fbfd1e None admin] RBAC: Authorizing `identity:get_user(user_id=76b6ddf40257432aa78fadccd4348cd5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:01.167850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf027ce9-79d5-404d-93b8-7f1ef1fbfd1e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:01.170014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 73/146] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:00 2022] GET /identity/v3/users/76b6ddf40257432aa78fadccd4348cd5 => generated 247 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:01.171115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5490-bfe6-4082-8bf6-532d9a69315e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.171436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5490-bfe6-4082-8bf6-532d9a69315e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.171715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5490-bfe6-4082-8bf6-532d9a69315e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.202403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8a3f5490-bfe6-4082-8bf6-532d9a69315e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:01.203986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8a3f5490-bfe6-4082-8bf6-532d9a69315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:01.236831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:01.237898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.238376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.238851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.239849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:01.240427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:01.242058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:01.262448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 74/147] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:01.263737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:01.277506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 74/148] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:01.278464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f5490-bfe6-4082-8bf6-532d9a69315e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:01.304462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:01.396993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:01.400426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f5490-bfe6-4082-8bf6-532d9a69315e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:01.408745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8eeda-6dff-4cec-975a-bf5145780251 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:01.443826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc3317-a36c-4c9e-b943-a0ae6fb2d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:01.444753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3317-a36c-4c9e-b943-a0ae6fb2d835 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.445026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a74cb8-ee27-4256-8f52-21d64f2529dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:01.445389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3317-a36c-4c9e-b943-a0ae6fb2d835 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.445846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3317-a36c-4c9e-b943-a0ae6fb2d835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.445961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a74cb8-ee27-4256-8f52-21d64f2529dd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.446180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a74cb8-ee27-4256-8f52-21d64f2529dd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.446457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a74cb8-ee27-4256-8f52-21d64f2529dd None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.447673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a74cb8-ee27-4256-8f52-21d64f2529dd None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:01.449395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a74cb8-ee27-4256-8f52-21d64f2529dd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:01.451949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 75/149] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:01 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:01.453393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3a742-dde2-4086-93a2-f5855746b27c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.453674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3a742-dde2-4086-93a2-f5855746b27c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.453948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3a742-dde2-4086-93a2-f5855746b27c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.459319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 76/150] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:01.460727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3a742-dde2-4086-93a2-f5855746b27c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:01.539897 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc3317-a36c-4c9e-b943-a0ae6fb2d835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:01.541522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc3317-a36c-4c9e-b943-a0ae6fb2d835 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:01.562810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:01.574789 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3a742-dde2-4086-93a2-f5855746b27c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:01.610472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cefce-177c-4bb1-b7f4-2238a9ffc02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:01.611652 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cefce-177c-4bb1-b7f4-2238a9ffc02b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.612109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cefce-177c-4bb1-b7f4-2238a9ffc02b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.612558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cefce-177c-4bb1-b7f4-2238a9ffc02b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.613895 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cefce-177c-4bb1-b7f4-2238a9ffc02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:01.615771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cefce-177c-4bb1-b7f4-2238a9ffc02b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:01.618435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 77/151] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:01.624415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cefce-177c-4bb1-b7f4-2238a9ffc02b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:01.661369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:01.674654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 75/152] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:01.676803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859d01d-f8c0-42d2-a991-fde31e14f7aa None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.677103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859d01d-f8c0-42d2-a991-fde31e14f7aa None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.677447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859d01d-f8c0-42d2-a991-fde31e14f7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.699051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d859d01d-f8c0-42d2-a991-fde31e14f7aa None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:01.700274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d859d01d-f8c0-42d2-a991-fde31e14f7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:01.734740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cefce-177c-4bb1-b7f4-2238a9ffc02b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:01.775862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188291f3-0c5b-4ae9-827d-b07c19541da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:01.776525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188291f3-0c5b-4ae9-827d-b07c19541da9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.776714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 76/153] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:01.787411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188291f3-0c5b-4ae9-827d-b07c19541da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.787411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188291f3-0c5b-4ae9-827d-b07c19541da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.787679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859d01d-f8c0-42d2-a991-fde31e14f7aa None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:01.814658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:01.854505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188291f3-0c5b-4ae9-827d-b07c19541da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:01.856034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188291f3-0c5b-4ae9-827d-b07c19541da9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:01.886617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859d01d-f8c0-42d2-a991-fde31e14f7aa None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:01.910243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:01.921993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5ea53-5204-4f36-872b-4f3b930f6aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:01.922601 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5ea53-5204-4f36-872b-4f3b930f6aef None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.922879 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5ea53-5204-4f36-872b-4f3b930f6aef None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.923148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5ea53-5204-4f36-872b-4f3b930f6aef None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.923880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5ea53-5204-4f36-872b-4f3b930f6aef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:01.925325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5ea53-5204-4f36-872b-4f3b930f6aef None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:01.945925 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 77/154] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:01 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:01.946724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5ea53-5204-4f36-872b-4f3b930f6aef None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:01.965985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 78/155] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:01.967371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fecf2b-d1c5-4b49-a63c-a2a0b839d815 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.967693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fecf2b-d1c5-4b49-a63c-a2a0b839d815 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.967952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fecf2b-d1c5-4b49-a63c-a2a0b839d815 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.969575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 79/156] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:01.972734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfaccc-6f89-4291-86d7-84d8f78667a7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:01.973526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfaccc-6f89-4291-86d7-84d8f78667a7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:01.973806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfaccc-6f89-4291-86d7-84d8f78667a7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:01.975439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 80/157] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:01.976140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfaccc-6f89-4291-86d7-84d8f78667a7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:02.049084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:02.088881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5ea53-5204-4f36-872b-4f3b930f6aef None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:02.098378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfaccc-6f89-4291-86d7-84d8f78667a7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:02.126033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aa1a4-38f0-47a7-98ba-d4ae1f4f3143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:02.126902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aa1a4-38f0-47a7-98ba-d4ae1f4f3143 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.127180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aa1a4-38f0-47a7-98ba-d4ae1f4f3143 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.127475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aa1a4-38f0-47a7-98ba-d4ae1f4f3143 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.128673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3aa1a4-38f0-47a7-98ba-d4ae1f4f3143 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:02.130152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3aa1a4-38f0-47a7-98ba-d4ae1f4f3143 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:02.132151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 78/158] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:01 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:02.132710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aa1a4-38f0-47a7-98ba-d4ae1f4f3143 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:02.145164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46285e1c-627e-4c99-9335-fc2c1ba90e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:02.145971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46285e1c-627e-4c99-9335-fc2c1ba90e33 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.146251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46285e1c-627e-4c99-9335-fc2c1ba90e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.146524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46285e1c-627e-4c99-9335-fc2c1ba90e33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.160205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46285e1c-627e-4c99-9335-fc2c1ba90e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-46285e1c-627e-4c99-9335-fc2c1ba90e33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:02.168200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:02.169145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:02.169902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 81/159] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:02.170768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8ca8a-2fbc-43b3-b41c-ec7dc2fcf6a4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.170902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8ca8a-2fbc-43b3-b41c-ec7dc2fcf6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.176367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8ca8a-2fbc-43b3-b41c-ec7dc2fcf6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.194813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:02.197905 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d1b8ca8a-2fbc-43b3-b41c-ec7dc2fcf6a4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:02.199466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d1b8ca8a-2fbc-43b3-b41c-ec7dc2fcf6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:02.263131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aa1a4-38f0-47a7-98ba-d4ae1f4f3143 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:02.299233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 82/160] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:02.299233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51be07e-0239-49cd-8661-e48ad6bb5728 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.299233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51be07e-0239-49cd-8661-e48ad6bb5728 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.299233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51be07e-0239-49cd-8661-e48ad6bb5728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.307897 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84b107-b76d-457c-9b78-11511723f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:02.309318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84b107-b76d-457c-9b78-11511723f289 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.309602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84b107-b76d-457c-9b78-11511723f289 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.309878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84b107-b76d-457c-9b78-11511723f289 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.311487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84b107-b76d-457c-9b78-11511723f289 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:02.312935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84b107-b76d-457c-9b78-11511723f289 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:02.315303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 79/161] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:02 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:02.315571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c51be07e-0239-49cd-8661-e48ad6bb5728 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:02.316080 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84b107-b76d-457c-9b78-11511723f289 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:02.316997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c51be07e-0239-49cd-8661-e48ad6bb5728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:02.374813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:02.403882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 83/162] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:02.405319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51be07e-0239-49cd-8661-e48ad6bb5728 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:02.463267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84b107-b76d-457c-9b78-11511723f289 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:02.482595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:02.513448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:02.514129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.514409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.514684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.516329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=20862a0ad40a41af9af9f940f3d4c544, user.id=76b6ddf40257432aa78fadccd4348cd5) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:02.516641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:02.518128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:02.524924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 80/163] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:02 2022] GET /identity/v3/role_assignments?user.id=76b6ddf40257432aa78fadccd4348cd5&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:02.525805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:02.549028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51be07e-0239-49cd-8661-e48ad6bb5728 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:02.591441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:02.592126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.592407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.592676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.593529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:02.593835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:02.595406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:02.605471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 84/164] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:02.606216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:02.651175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36a57a-94bd-4ad7-b51f-078b579410b7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:02.673452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:02.696453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:02.697142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.697492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.697772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.698928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] RBAC: Adding query filter params (role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=20862a0ad40a41af9af9f940f3d4c544, user.id=9f5f4a174ee24efbbdabdbbd9715f21e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:02.700682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:02.702233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:02.713698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 81/165] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:02 2022] GET /identity/v3/role_assignments?user.id=9f5f4a174ee24efbbdabdbbd9715f21e&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=798b7bcb53684ab49ae899f8902f444a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:02.714313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:02.795932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624d36e-be39-48d6-b183-07a2887b719e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:02.819693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:02.860118 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ead838-bf04-47fa-9d48-f4360c419bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:02.861574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ead838-bf04-47fa-9d48-f4360c419bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:02.861864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ead838-bf04-47fa-9d48-f4360c419bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:02.862144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ead838-bf04-47fa-9d48-f4360c419bf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:02.876355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ead838-bf04-47fa-9d48-f4360c419bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b3ead838-bf04-47fa-9d48-f4360c419bf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:02.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:02.886780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 85/166] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:02.892207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ead838-bf04-47fa-9d48-f4360c419bf7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:02.933995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ab950-0b8b-4b8c-8d59-de453de39305 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:02.999464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28894b41-a51d-4126-8195-fda24eef247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.000266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28894b41-a51d-4126-8195-fda24eef247b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.000555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28894b41-a51d-4126-8195-fda24eef247b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.000831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28894b41-a51d-4126-8195-fda24eef247b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.028300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28894b41-a51d-4126-8195-fda24eef247b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-28894b41-a51d-4126-8195-fda24eef247b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:03.037928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:03.040171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:03.044964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:03.046387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 82/167] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:03.051346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28894b41-a51d-4126-8195-fda24eef247b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:03.109762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ead838-bf04-47fa-9d48-f4360c419bf7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:03.111652 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:03.153017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54cb8-f573-4114-9dc6-7ceca89c0150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.154284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54cb8-f573-4114-9dc6-7ceca89c0150 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.154880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54cb8-f573-4114-9dc6-7ceca89c0150 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.155405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54cb8-f573-4114-9dc6-7ceca89c0150 None admin] PATH_INFO: `/v3/users/a8fa1faa859d4ccdac4c6ad255b8cf42` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.156908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c54cb8-f573-4114-9dc6-7ceca89c0150 None admin] RBAC: Authorizing `identity:get_user(user_id=a8fa1faa859d4ccdac4c6ad255b8cf42)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:03.158979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c54cb8-f573-4114-9dc6-7ceca89c0150 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:03.161290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 86/168] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:02 2022] GET /identity/v3/users/a8fa1faa859d4ccdac4c6ad255b8cf42 => generated 289 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:03.162472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54cb8-f573-4114-9dc6-7ceca89c0150 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:03.182457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28894b41-a51d-4126-8195-fda24eef247b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:03.184213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:03.219715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.220393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.220680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.220959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.221723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:03.222031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:03.223458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:03.233642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 83/169] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:03.234297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:03.294070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54cb8-f573-4114-9dc6-7ceca89c0150 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:03.335406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.337434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.337912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.338839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.340218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:03.340848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:03.342772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:03.354563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 87/170] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:03.355987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:03.368362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fc09d-b116-4e32-ae54-3156a9d2559e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:03.398064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:03.423017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131339df-3fdf-4dfd-b609-e06e0195a86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.423929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131339df-3fdf-4dfd-b609-e06e0195a86a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.424209 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131339df-3fdf-4dfd-b609-e06e0195a86a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.424482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131339df-3fdf-4dfd-b609-e06e0195a86a None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.426828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131339df-3fdf-4dfd-b609-e06e0195a86a None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:03.428402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131339df-3fdf-4dfd-b609-e06e0195a86a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:03.430137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 84/171] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:03 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:03.430910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131339df-3fdf-4dfd-b609-e06e0195a86a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:03.483252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:03.519080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa39e3-2bc7-4b30-8cdd-37503a0f43a5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:03.566796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577c937-54d4-4469-9b62-9f4538a14c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.568750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c937-54d4-4469-9b62-9f4538a14c83 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.569034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c937-54d4-4469-9b62-9f4538a14c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.569300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c937-54d4-4469-9b62-9f4538a14c83 None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.571154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577c937-54d4-4469-9b62-9f4538a14c83 None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:03.572661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577c937-54d4-4469-9b62-9f4538a14c83 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:03.575493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 88/172] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:03 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:03.579378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577c937-54d4-4469-9b62-9f4538a14c83 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:03.637889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131339df-3fdf-4dfd-b609-e06e0195a86a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:03.696961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c70c4-7c8c-404f-b087-afd742f455df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.698206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c70c4-7c8c-404f-b087-afd742f455df None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.698759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c70c4-7c8c-404f-b087-afd742f455df None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.699374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c70c4-7c8c-404f-b087-afd742f455df None admin] PATH_INFO: `/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.703980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c70c4-7c8c-404f-b087-afd742f455df None admin] RBAC: Authorizing `identity:get_user(user_id=9f5f4a174ee24efbbdabdbbd9715f21e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:03.705672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c70c4-7c8c-404f-b087-afd742f455df None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:03.707559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 85/173] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:03 2022] GET /identity/v3/users/9f5f4a174ee24efbbdabdbbd9715f21e => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:03.708519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c70c4-7c8c-404f-b087-afd742f455df None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:03.760205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:03.762984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577c937-54d4-4469-9b62-9f4538a14c83 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:03.831733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:03.834733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea2f24-d65a-48ca-9629-062a57656862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.835625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea2f24-d65a-48ca-9629-062a57656862 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.835904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea2f24-d65a-48ca-9629-062a57656862 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.836193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea2f24-d65a-48ca-9629-062a57656862 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544/users/a8fa1faa859d4ccdac4c6ad255b8cf42/roles/e7d8e4fc6f35425ab4e5c87ed577de1f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.837579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c70c4-7c8c-404f-b087-afd742f455df None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:03.844588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea2f24-d65a-48ca-9629-062a57656862 None admin] RBAC: Authorizing `identity:create_grant(project_id=20862a0ad40a41af9af9f940f3d4c544, user_id=a8fa1faa859d4ccdac4c6ad255b8cf42, role_id=e7d8e4fc6f35425ab4e5c87ed577de1f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:03.846071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea2f24-d65a-48ca-9629-062a57656862 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:03.873950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 89/174] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:03 2022] PUT /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544/users/a8fa1faa859d4ccdac4c6ad255b8cf42/roles/e7d8e4fc6f35425ab4e5c87ed577de1f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:03.874716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea2f24-d65a-48ca-9629-062a57656862 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:03.877757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd4e37-d7c0-4885-a20f-ef4f2ffb3b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:03.878827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4e37-d7c0-4885-a20f-ef4f2ffb3b63 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:03.879363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4e37-d7c0-4885-a20f-ef4f2ffb3b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:03.879872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4e37-d7c0-4885-a20f-ef4f2ffb3b63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:03.963973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd4e37-d7c0-4885-a20f-ef4f2ffb3b63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:03.965882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd4e37-d7c0-4885-a20f-ef4f2ffb3b63 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:04.076088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:04.081231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 86/175] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:04.087128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea2f24-d65a-48ca-9629-062a57656862 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:04.088421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd4e37-d7c0-4885-a20f-ef4f2ffb3b63 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:04.176966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b12fd4-8c3b-4c3a-b8e3-31f21b4ae97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:04.177707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b12fd4-8c3b-4c3a-b8e3-31f21b4ae97b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:04.178560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b12fd4-8c3b-4c3a-b8e3-31f21b4ae97b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:04.178986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b12fd4-8c3b-4c3a-b8e3-31f21b4ae97b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:04.203409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd4e37-d7c0-4885-a20f-ef4f2ffb3b63 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:04.238679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:04.245374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0d0d-8afa-4dad-9e8c-16613be9f829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:04.246505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0d0d-8afa-4dad-9e8c-16613be9f829 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:04.247298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0d0d-8afa-4dad-9e8c-16613be9f829 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:04.247714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0d0d-8afa-4dad-9e8c-16613be9f829 None admin] PATH_INFO: `/v3/projects/6aa34392639c42f1a4cb470639efb105` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:04.249650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda0d0d-8afa-4dad-9e8c-16613be9f829 None admin] RBAC: Authorizing `identity:get_project(project_id=6aa34392639c42f1a4cb470639efb105)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:04.251461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda0d0d-8afa-4dad-9e8c-16613be9f829 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:04.253385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 87/176] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:04 2022] GET /identity/v3/projects/6aa34392639c42f1a4cb470639efb105 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:04.260225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0d0d-8afa-4dad-9e8c-16613be9f829 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:04.324284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b12fd4-8c3b-4c3a-b8e3-31f21b4ae97b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:04.326197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b12fd4-8c3b-4c3a-b8e3-31f21b4ae97b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:04.333124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:04.370610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0d0d-8afa-4dad-9e8c-16613be9f829 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:04.409393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:04.412680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:04.413114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:04.413555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:04.414823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] RBAC: Adding query filter params (role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=6aa34392639c42f1a4cb470639efb105, user.id=9f5f4a174ee24efbbdabdbbd9715f21e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:04.415503 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:04.417897 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:04.431243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 88/177] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:04 2022] GET /identity/v3/role_assignments?user.id=9f5f4a174ee24efbbdabdbbd9715f21e&scope.project.id=6aa34392639c42f1a4cb470639efb105&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:04.522859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 90/178] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:04.531142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:04.656807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ab2b0-5531-42e6-a8e2-5a8708e51ed3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:04.688641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:04.694259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b2c22-01cf-47fa-b1cf-290e38c1ea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:04.695425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b2c22-01cf-47fa-b1cf-290e38c1ea48 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:04.695886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b2c22-01cf-47fa-b1cf-290e38c1ea48 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:04.696345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b2c22-01cf-47fa-b1cf-290e38c1ea48 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:04.697708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b2c22-01cf-47fa-b1cf-290e38c1ea48 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:04.699357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b2c22-01cf-47fa-b1cf-290e38c1ea48 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:04.701225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 89/179] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:04 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:04.707513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b12fd4-8c3b-4c3a-b8e3-31f21b4ae97b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:04.723157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c65d86-5950-4238-ab27-e04f8b52b11c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:04.723962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c65d86-5950-4238-ab27-e04f8b52b11c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:04.724385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c65d86-5950-4238-ab27-e04f8b52b11c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:04.728848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 90/180] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:04.734690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075cce-c654-403a-b202-c1f35512fed0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:04.735197 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075cce-c654-403a-b202-c1f35512fed0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:04.735645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075cce-c654-403a-b202-c1f35512fed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:04.765879 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-47075cce-c654-403a-b202-c1f35512fed0 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:04.768690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-47075cce-c654-403a-b202-c1f35512fed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:04.833561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:04.912627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 91/181] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:04.918379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b12fd4-8c3b-4c3a-b8e3-31f21b4ae97b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:04.947890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47075cce-c654-403a-b202-c1f35512fed0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:04.977782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:04.978783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:04.979229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:04.979670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:04.980749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] RBAC: Adding query filter params (role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=20862a0ad40a41af9af9f940f3d4c544, user.id=9f5f4a174ee24efbbdabdbbd9715f21e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:04.981204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:04.983074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:04.994175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 91/182] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:04 2022] GET /identity/v3/role_assignments?user.id=9f5f4a174ee24efbbdabdbbd9715f21e&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:05.073322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47075cce-c654-403a-b202-c1f35512fed0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:05.110242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:05.114969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddd625-e761-4d27-9fca-563e5d53ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:05.115950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddd625-e761-4d27-9fca-563e5d53ae33 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.116257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddd625-e761-4d27-9fca-563e5d53ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.116558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddd625-e761-4d27-9fca-563e5d53ae33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.129652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddd625-e761-4d27-9fca-563e5d53ae33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-26ddd625-e761-4d27-9fca-563e5d53ae33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:05.138216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:05.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:05.139989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 92/183] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:05.146850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:05.211187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489a1b-ca1f-41d3-a87d-d9b05a53fd10 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.211434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489a1b-ca1f-41d3-a87d-d9b05a53fd10 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.211716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489a1b-ca1f-41d3-a87d-d9b05a53fd10 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.213249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 93/184] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:05.220992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0ce4d-2e47-4996-b4ab-e8e34f50a041 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.221339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0ce4d-2e47-4996-b4ab-e8e34f50a041 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.221624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0ce4d-2e47-4996-b4ab-e8e34f50a041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.264500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7ac0ce4d-2e47-4996-b4ab-e8e34f50a041 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:05.265761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7ac0ce4d-2e47-4996-b4ab-e8e34f50a041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:05.292910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b937f-a22a-4ebc-af0b-119c2100e9ef None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:05.317601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:05.330729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:05.331782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.332290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.332752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.334283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:05.334854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:05.336531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:05.345348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 92/185] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:05.351247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 94/186] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:05.352457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:05.377764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5d8b-0f95-44fb-b294-91869f4dd2ef None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.378098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5d8b-0f95-44fb-b294-91869f4dd2ef None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.378407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5d8b-0f95-44fb-b294-91869f4dd2ef None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.379963 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 95/187] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:05.380522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb5d8b-0f95-44fb-b294-91869f4dd2ef None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:05.473117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71790c94-f94c-4bc1-905b-848d453ee907 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:05.501592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb5d8b-0f95-44fb-b294-91869f4dd2ef None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:05.518138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d56b8-b740-4b81-ae17-bd4b74d42f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:05.519076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d56b8-b740-4b81-ae17-bd4b74d42f70 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.519409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d56b8-b740-4b81-ae17-bd4b74d42f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.519686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d56b8-b740-4b81-ae17-bd4b74d42f70 None admin] PATH_INFO: `/v3/users/a4c718641b5e42f6ba3e7d5819367430` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.523497 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:05.534751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d56b8-b740-4b81-ae17-bd4b74d42f70 None admin] RBAC: Authorizing `identity:get_user(user_id=a4c718641b5e42f6ba3e7d5819367430)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:05.536379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d56b8-b740-4b81-ae17-bd4b74d42f70 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:05.537795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cb4a5-4cd3-4e1a-be19-a16859567e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:05.542948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 93/188] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:05 2022] GET /identity/v3/users/a4c718641b5e42f6ba3e7d5819367430 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:05.543074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb4a5-4cd3-4e1a-be19-a16859567e94 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.543074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb4a5-4cd3-4e1a-be19-a16859567e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.543074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb4a5-4cd3-4e1a-be19-a16859567e94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.543318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be39dd0-0254-4011-a529-0826c47f1f91 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.543318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be39dd0-0254-4011-a529-0826c47f1f91 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.543318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be39dd0-0254-4011-a529-0826c47f1f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.555613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cb4a5-4cd3-4e1a-be19-a16859567e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:05.559178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2be39dd0-0254-4011-a529-0826c47f1f91 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:05.560402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2be39dd0-0254-4011-a529-0826c47f1f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-047cb4a5-4cd3-4e1a-be19-a16859567e94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:05.564050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:05.564915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:05.565749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 96/189] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:05.566408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cb4a5-4cd3-4e1a-be19-a16859567e94 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:05.608480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:05.636430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 94/190] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:05.644115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be39dd0-0254-4011-a529-0826c47f1f91 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:05.677849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cb4a5-4cd3-4e1a-be19-a16859567e94 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:05.715914 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff859525-163b-48cb-b652-b93f3410e2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:05.716605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff859525-163b-48cb-b652-b93f3410e2ce None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.716964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff859525-163b-48cb-b652-b93f3410e2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.717242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff859525-163b-48cb-b652-b93f3410e2ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.750965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be39dd0-0254-4011-a529-0826c47f1f91 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:05.770131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:05.787244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:05.788281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.788661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.789484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.790344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:05.790751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:05.791014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff859525-163b-48cb-b652-b93f3410e2ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:05.792383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:05.792572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff859525-163b-48cb-b652-b93f3410e2ce None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:05.804965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 95/191] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:05.804965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:05.842880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:05.909753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 97/192] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:05.909753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff859525-163b-48cb-b652-b93f3410e2ce None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:05.918060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109ffc4-686f-478f-9598-d27e1a2320b0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:05.953344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ada2c-6ce6-4760-9320-88eb176f86bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:05.954142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ada2c-6ce6-4760-9320-88eb176f86bd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:05.954423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ada2c-6ce6-4760-9320-88eb176f86bd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:05.954692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ada2c-6ce6-4760-9320-88eb176f86bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:05.955881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ada2c-6ce6-4760-9320-88eb176f86bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:05.957343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ada2c-6ce6-4760-9320-88eb176f86bd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:05.959228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 96/193] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:05.959851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ada2c-6ce6-4760-9320-88eb176f86bd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:05.995009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:06.016632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff859525-163b-48cb-b652-b93f3410e2ce None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:06.050915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f92621-dcb6-4d59-b775-958973008bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:06.051850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f92621-dcb6-4d59-b775-958973008bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.052105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f92621-dcb6-4d59-b775-958973008bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.052378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f92621-dcb6-4d59-b775-958973008bb5 None admin] PATH_INFO: `/v3/users/a4c718641b5e42f6ba3e7d5819367430` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.053826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f92621-dcb6-4d59-b775-958973008bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=a4c718641b5e42f6ba3e7d5819367430)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:06.055303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f92621-dcb6-4d59-b775-958973008bb5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:06.056935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 98/194] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:05 2022] GET /identity/v3/users/a4c718641b5e42f6ba3e7d5819367430 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:06.062211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f92621-dcb6-4d59-b775-958973008bb5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:06.066520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ada2c-6ce6-4760-9320-88eb176f86bd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:06.069860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:06.105032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec3eb9-583a-4cbe-a2ab-4dcd6acb16bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:06.106012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3eb9-583a-4cbe-a2ab-4dcd6acb16bb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.106294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3eb9-583a-4cbe-a2ab-4dcd6acb16bb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.106569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3eb9-583a-4cbe-a2ab-4dcd6acb16bb None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.107885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec3eb9-583a-4cbe-a2ab-4dcd6acb16bb None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:06.109378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec3eb9-583a-4cbe-a2ab-4dcd6acb16bb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:06.111560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 97/195] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:05 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:06.112153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec3eb9-583a-4cbe-a2ab-4dcd6acb16bb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:06.166899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f92621-dcb6-4d59-b775-958973008bb5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:06.201991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fa6eb-9558-4e7b-a9ec-b42b39aabf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:06.202657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fa6eb-9558-4e7b-a9ec-b42b39aabf87 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.202937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fa6eb-9558-4e7b-a9ec-b42b39aabf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.203173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fa6eb-9558-4e7b-a9ec-b42b39aabf87 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.203951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fa6eb-9558-4e7b-a9ec-b42b39aabf87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:06.205396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fa6eb-9558-4e7b-a9ec-b42b39aabf87 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:06.221040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec3eb9-583a-4cbe-a2ab-4dcd6acb16bb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:06.227997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 99/196] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:06 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:06.240414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fa6eb-9558-4e7b-a9ec-b42b39aabf87 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:06.245584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:06.261641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dba2fe-0745-436b-a8ee-f7df82c60922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:06.262567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba2fe-0745-436b-a8ee-f7df82c60922 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.262990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba2fe-0745-436b-a8ee-f7df82c60922 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.263340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba2fe-0745-436b-a8ee-f7df82c60922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.313515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:06.344961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dba2fe-0745-436b-a8ee-f7df82c60922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:06.346677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dba2fe-0745-436b-a8ee-f7df82c60922 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:06.375727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fa6eb-9558-4e7b-a9ec-b42b39aabf87 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:06.415821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:06.416727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.417162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.417434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.418274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=e44154970e4b4efda8f5cd260edbc63e, user.id=a4c718641b5e42f6ba3e7d5819367430) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:06.418589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:06.420895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:06.427956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 100/197] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:06 2022] GET /identity/v3/role_assignments?user.id=a4c718641b5e42f6ba3e7d5819367430&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:06.460579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 98/198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:06.466389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:06.558989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:06.599842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b073db5-d646-4d83-8b20-8ce29fa5fa03 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:06.621435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8cf06-fd89-4a37-b261-14de896dd1c2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.622280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8cf06-fd89-4a37-b261-14de896dd1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.622280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8cf06-fd89-4a37-b261-14de896dd1c2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.623905 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 99/199] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:06.629184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d620a7-28cb-4012-92e4-70bfe67858a0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.629424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d620a7-28cb-4012-92e4-70bfe67858a0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.629693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d620a7-28cb-4012-92e4-70bfe67858a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.642328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022525a-651a-4091-b886-005b80c37285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:06.643200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022525a-651a-4091-b886-005b80c37285 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.643516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022525a-651a-4091-b886-005b80c37285 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.643783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022525a-651a-4091-b886-005b80c37285 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.644985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022525a-651a-4091-b886-005b80c37285 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:06.646453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022525a-651a-4091-b886-005b80c37285 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:06.648243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 101/200] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:06 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:06.652989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-33d620a7-28cb-4012-92e4-70bfe67858a0 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:06.654010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022525a-651a-4091-b886-005b80c37285 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:06.654208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-33d620a7-28cb-4012-92e4-70bfe67858a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:06.732342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 100/201] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:06.758671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d620a7-28cb-4012-92e4-70bfe67858a0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:06.760162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022525a-651a-4091-b886-005b80c37285 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:06.760942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:06.795868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:06.796702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.796904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.797169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.798001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] RBAC: Adding query filter params (role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=e44154970e4b4efda8f5cd260edbc63e, user.id=a4c718641b5e42f6ba3e7d5819367430) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:06.798308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:06.799811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:06.805941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:06.811555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 102/202] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:06 2022] GET /identity/v3/role_assignments?user.id=a4c718641b5e42f6ba3e7d5819367430&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=798b7bcb53684ab49ae899f8902f444a => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:06.924214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d620a7-28cb-4012-92e4-70bfe67858a0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:06.970447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b50792-a00c-4678-b06c-369410d9ab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:06.971720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50792-a00c-4678-b06c-369410d9ab61 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:06.972203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50792-a00c-4678-b06c-369410d9ab61 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:06.972670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50792-a00c-4678-b06c-369410d9ab61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:06.984391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b50792-a00c-4678-b06c-369410d9ab61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b2b50792-a00c-4678-b06c-369410d9ab61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:06.992863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:06.993857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:06.994648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:06.997382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 101/203] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:07.003147 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:07.117961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5438-a52a-4a84-b9b8-988c28d72873 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:07.158376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:07.159465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:07.159964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:07.160431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:07.161383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:07.161880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:07.163532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:07.172744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 103/204] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:07.178578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b50792-a00c-4678-b06c-369410d9ab61 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:07.269054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:07.321378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b50792-a00c-4678-b06c-369410d9ab61 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:07.367970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee5954-3dd4-4872-bd22-ee9cbd0f43b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:07.368860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5954-3dd4-4872-bd22-ee9cbd0f43b5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:07.369144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5954-3dd4-4872-bd22-ee9cbd0f43b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:07.369418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5954-3dd4-4872-bd22-ee9cbd0f43b5 None admin] PATH_INFO: `/v3/users/a8fa1faa859d4ccdac4c6ad255b8cf42` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:07.375940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee5954-3dd4-4872-bd22-ee9cbd0f43b5 None admin] RBAC: Authorizing `identity:get_user(user_id=a8fa1faa859d4ccdac4c6ad255b8cf42)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:07.377553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee5954-3dd4-4872-bd22-ee9cbd0f43b5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:07.379414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 102/205] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:07 2022] GET /identity/v3/users/a8fa1faa859d4ccdac4c6ad255b8cf42 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:07.384761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:07.463646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:07.516202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120762-d5d9-4378-8d0f-cd7ce03682cd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:07.575917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d8b0f-47bc-4d25-b820-54f42ed20fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:07.576700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d8b0f-47bc-4d25-b820-54f42ed20fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:07.576897 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d8b0f-47bc-4d25-b820-54f42ed20fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:07.577178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d8b0f-47bc-4d25-b820-54f42ed20fc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:07.662907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d8b0f-47bc-4d25-b820-54f42ed20fc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:07.664400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d8b0f-47bc-4d25-b820-54f42ed20fc4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:07.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:07.774332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 104/206] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:07.779625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee5954-3dd4-4872-bd22-ee9cbd0f43b5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:07.913602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee5954-3dd4-4872-bd22-ee9cbd0f43b5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:07.943624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:07.948983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de199f-6955-4a57-a43f-bde7b0817ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:07.949867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de199f-6955-4a57-a43f-bde7b0817ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:07.950153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de199f-6955-4a57-a43f-bde7b0817ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:07.950428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de199f-6955-4a57-a43f-bde7b0817ce4 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:07.951655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de199f-6955-4a57-a43f-bde7b0817ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:07.953135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de199f-6955-4a57-a43f-bde7b0817ce4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:07.955151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 103/207] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:07 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:07.961214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d8b0f-47bc-4d25-b820-54f42ed20fc4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:08.101357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d8b0f-47bc-4d25-b820-54f42ed20fc4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:08.146073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a362a02-c829-4617-bd3c-ca688d3c683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:08.146893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a362a02-c829-4617-bd3c-ca688d3c683a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:08.147177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a362a02-c829-4617-bd3c-ca688d3c683a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:08.147471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a362a02-c829-4617-bd3c-ca688d3c683a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:08.148303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a362a02-c829-4617-bd3c-ca688d3c683a None admin] RBAC: Adding query filter params (role.id=e7d8e4fc6f35425ab4e5c87ed577de1f, scope.project.id=20862a0ad40a41af9af9f940f3d4c544, user.id=a8fa1faa859d4ccdac4c6ad255b8cf42) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:08.148609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a362a02-c829-4617-bd3c-ca688d3c683a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:08.150095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a362a02-c829-4617-bd3c-ca688d3c683a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:08.156303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 105/208] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:07 2022] GET /identity/v3/role_assignments?user.id=a8fa1faa859d4ccdac4c6ad255b8cf42&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=e7d8e4fc6f35425ab4e5c87ed577de1f => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:08.887391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0712-fa11-4de7-bee3-bfa505e0a2ab None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:08.887926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0712-fa11-4de7-bee3-bfa505e0a2ab None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:08.887926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0712-fa11-4de7-bee3-bfa505e0a2ab None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:08.889579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 104/209] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:08.894982 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746e6d9-f156-424c-9cc2-7ffeec5cfe3e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:08.895426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746e6d9-f156-424c-9cc2-7ffeec5cfe3e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:08.895596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746e6d9-f156-424c-9cc2-7ffeec5cfe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:08.912457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e746e6d9-f156-424c-9cc2-7ffeec5cfe3e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:08.913710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e746e6d9-f156-424c-9cc2-7ffeec5cfe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:08.934043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:08.981992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 106/210] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:08.982179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d4762-91ec-4b97-b23a-a020f56c056a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:08.982409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d4762-91ec-4b97-b23a-a020f56c056a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:08.982661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d4762-91ec-4b97-b23a-a020f56c056a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:08.984316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 105/211] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:08.989352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004276f-5413-46a1-b26f-5a7a62f8783e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:08.989644 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004276f-5413-46a1-b26f-5a7a62f8783e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:08.989920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004276f-5413-46a1-b26f-5a7a62f8783e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.005634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ee58-589b-4f30-83b0-8f6f7cc91446 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.006016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ee58-589b-4f30-83b0-8f6f7cc91446 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.006357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ee58-589b-4f30-83b0-8f6f7cc91446 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.007284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d004276f-5413-46a1-b26f-5a7a62f8783e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:09.008297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 106/212] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:09.008493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d004276f-5413-46a1-b26f-5a7a62f8783e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:09.012554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ee58-589b-4f30-83b0-8f6f7cc91446 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.075728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 107/213] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:09.076862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf59a8a-8242-4060-844c-88afee7f7f95 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.077150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf59a8a-8242-4060-844c-88afee7f7f95 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.077400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf59a8a-8242-4060-844c-88afee7f7f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.101370 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fcf59a8a-8242-4060-844c-88afee7f7f95 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:09.102929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fcf59a8a-8242-4060-844c-88afee7f7f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:09.127365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ee58-589b-4f30-83b0-8f6f7cc91446 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.161982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16008967-5c21-408e-92be-de6734cc962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.162804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008967-5c21-408e-92be-de6734cc962b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.163108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008967-5c21-408e-92be-de6734cc962b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.163428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008967-5c21-408e-92be-de6734cc962b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.174011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16008967-5c21-408e-92be-de6734cc962b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:09.179373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 108/214] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:09.179373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf59a8a-8242-4060-844c-88afee7f7f95 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-16008967-5c21-408e-92be-de6734cc962b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:09.182082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:09.182939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:09.187500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 107/215] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:09.189242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16008967-5c21-408e-92be-de6734cc962b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.204610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:09.254835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:09.290507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf59a8a-8242-4060-844c-88afee7f7f95 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.297986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16008967-5c21-408e-92be-de6734cc962b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.326115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7f868-405a-4b6c-8d96-56de65f1ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.326916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7f868-405a-4b6c-8d96-56de65f1ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.327174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7f868-405a-4b6c-8d96-56de65f1ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.327508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7f868-405a-4b6c-8d96-56de65f1ebd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.328721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7f868-405a-4b6c-8d96-56de65f1ebd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:09.330185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7f868-405a-4b6c-8d96-56de65f1ebd5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:09.332078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f9c77-e908-4067-ba15-4fbf7c34bbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.332226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 109/216] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:09.332511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9c77-e908-4067-ba15-4fbf7c34bbdb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.332634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7f868-405a-4b6c-8d96-56de65f1ebd5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.332756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9c77-e908-4067-ba15-4fbf7c34bbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.333045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9c77-e908-4067-ba15-4fbf7c34bbdb None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.333822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f9c77-e908-4067-ba15-4fbf7c34bbdb None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:09.334329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f9c77-e908-4067-ba15-4fbf7c34bbdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:09.335929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f9c77-e908-4067-ba15-4fbf7c34bbdb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:09.345133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 108/217] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:09.346228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68c849-1644-4a30-a830-f880282a64b2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.346512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68c849-1644-4a30-a830-f880282a64b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.346799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68c849-1644-4a30-a830-f880282a64b2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.348039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 109/218] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:09.348715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68c849-1644-4a30-a830-f880282a64b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.421111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:09.437548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7f868-405a-4b6c-8d96-56de65f1ebd5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.449415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68c849-1644-4a30-a830-f880282a64b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.469331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:09.471105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632906a0-63cf-4286-bacd-176e601f7c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.472067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632906a0-63cf-4286-bacd-176e601f7c4b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.472402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632906a0-63cf-4286-bacd-176e601f7c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.472720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632906a0-63cf-4286-bacd-176e601f7c4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.482441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632906a0-63cf-4286-bacd-176e601f7c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:09.484319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dd7e0-1775-4b5c-b272-eecc25bee382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.484996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dd7e0-1775-4b5c-b272-eecc25bee382 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.485282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dd7e0-1775-4b5c-b272-eecc25bee382 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.485561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dd7e0-1775-4b5c-b272-eecc25bee382 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.486283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37dd7e0-1775-4b5c-b272-eecc25bee382 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:09.487800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37dd7e0-1775-4b5c-b272-eecc25bee382 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-632906a0-63cf-4286-bacd-176e601f7c4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:09.490096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:09.491187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:09.492281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 110/219] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:09.493069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632906a0-63cf-4286-bacd-176e601f7c4b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.511499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 110/220] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:09 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:09.520795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6e005-eb25-4679-8583-c2ba794e9c4b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.521086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6e005-eb25-4679-8583-c2ba794e9c4b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.521359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6e005-eb25-4679-8583-c2ba794e9c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.539952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c2f6e005-eb25-4679-8583-c2ba794e9c4b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:09.541374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c2f6e005-eb25-4679-8583-c2ba794e9c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:09.623229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632906a0-63cf-4286-bacd-176e601f7c4b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.625025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 111/221] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:09.625932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6e005-eb25-4679-8583-c2ba794e9c4b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.673863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565cb95-7006-4106-93f4-1396b0f005c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.675096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565cb95-7006-4106-93f4-1396b0f005c9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.675618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565cb95-7006-4106-93f4-1396b0f005c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.675900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565cb95-7006-4106-93f4-1396b0f005c9 None admin] PATH_INFO: `/v3/users/b84a8f049f88417596e197dd574ac961` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.693255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565cb95-7006-4106-93f4-1396b0f005c9 None admin] RBAC: Authorizing `identity:get_user(user_id=b84a8f049f88417596e197dd574ac961)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:09.695419 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565cb95-7006-4106-93f4-1396b0f005c9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:09.701214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 111/222] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/users/b84a8f049f88417596e197dd574ac961 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:09.701214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565cb95-7006-4106-93f4-1396b0f005c9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.701214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:09.739552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6e005-eb25-4679-8583-c2ba794e9c4b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.751586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:09.788978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.789677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.789957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.790266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.790999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:09.791352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:09.792779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:09.805222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 112/223] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:09.806259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.826518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565cb95-7006-4106-93f4-1396b0f005c9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.864551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e3089-30c0-4d23-bb18-8b2e95d4e7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.865507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e3089-30c0-4d23-bb18-8b2e95d4e7dd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.865890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e3089-30c0-4d23-bb18-8b2e95d4e7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.866338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e3089-30c0-4d23-bb18-8b2e95d4e7dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:09.877219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e3089-30c0-4d23-bb18-8b2e95d4e7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-024e3089-30c0-4d23-bb18-8b2e95d4e7dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:09.887052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:09.888105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:09.888899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 112/224] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:09.889552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e3089-30c0-4d23-bb18-8b2e95d4e7dd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:09.927912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337d3bd-3216-4fd6-823d-b6ff380da682 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:09.965585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db42e2d-7ebe-4eb7-885d-7fba57fa25b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:09.966487 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db42e2d-7ebe-4eb7-885d-7fba57fa25b8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:09.966911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db42e2d-7ebe-4eb7-885d-7fba57fa25b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:09.967406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:09.967849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db42e2d-7ebe-4eb7-885d-7fba57fa25b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.005320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:10.011773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e3089-30c0-4d23-bb18-8b2e95d4e7dd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.040026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db42e2d-7ebe-4eb7-885d-7fba57fa25b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.041849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db42e2d-7ebe-4eb7-885d-7fba57fa25b8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.047669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d8b23-ef8f-4f86-a185-3c23692fec6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.048569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d8b23-ef8f-4f86-a185-3c23692fec6a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.048848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d8b23-ef8f-4f86-a185-3c23692fec6a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.049120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d8b23-ef8f-4f86-a185-3c23692fec6a None admin] PATH_INFO: `/v3/users/a4c718641b5e42f6ba3e7d5819367430` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.050327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d8b23-ef8f-4f86-a185-3c23692fec6a None admin] RBAC: Authorizing `identity:get_user(user_id=a4c718641b5e42f6ba3e7d5819367430)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.052034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d8b23-ef8f-4f86-a185-3c23692fec6a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.058330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 113/225] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/users/a4c718641b5e42f6ba3e7d5819367430 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:10.058330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d8b23-ef8f-4f86-a185-3c23692fec6a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:10.152755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 113/226] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:10.154024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db42e2d-7ebe-4eb7-885d-7fba57fa25b8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:10.161188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d8b23-ef8f-4f86-a185-3c23692fec6a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.197373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.197997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.198277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.198548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.199569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:10.199905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.201387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.207625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:10.212862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 114/227] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:10.213755 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:10.226743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:10.261901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db42e2d-7ebe-4eb7-885d-7fba57fa25b8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.296794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76266b0-44e3-4a78-a7d4-84ec09419cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.297981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76266b0-44e3-4a78-a7d4-84ec09419cad None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.298360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76266b0-44e3-4a78-a7d4-84ec09419cad None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.298974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76266b0-44e3-4a78-a7d4-84ec09419cad None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.300129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76266b0-44e3-4a78-a7d4-84ec09419cad None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.301697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76266b0-44e3-4a78-a7d4-84ec09419cad None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.303615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 114/228] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:10 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:10.304469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76266b0-44e3-4a78-a7d4-84ec09419cad None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:10.326800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3061d-8f31-4df8-94c0-5c6ac9f1ef92 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.364781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe40c-7ef3-4c65-87d1-48337ab79f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.365937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe40c-7ef3-4c65-87d1-48337ab79f23 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.366438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe40c-7ef3-4c65-87d1-48337ab79f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.366921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe40c-7ef3-4c65-87d1-48337ab79f23 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.368430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fe40c-7ef3-4c65-87d1-48337ab79f23 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.370139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fe40c-7ef3-4c65-87d1-48337ab79f23 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.372584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 115/229] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:10 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:10.373652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe40c-7ef3-4c65-87d1-48337ab79f23 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:10.415352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76266b0-44e3-4a78-a7d4-84ec09419cad None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.438540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:10.451138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d33adf-4e12-48b4-bafe-e194cbce5078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.452086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33adf-4e12-48b4-bafe-e194cbce5078 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.452434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33adf-4e12-48b4-bafe-e194cbce5078 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.452776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33adf-4e12-48b4-bafe-e194cbce5078 None admin] PATH_INFO: `/v3/users/a4c718641b5e42f6ba3e7d5819367430` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.454485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d33adf-4e12-48b4-bafe-e194cbce5078 None admin] RBAC: Authorizing `identity:get_user(user_id=a4c718641b5e42f6ba3e7d5819367430)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.456146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d33adf-4e12-48b4-bafe-e194cbce5078 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.457879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 115/230] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:10 2022] GET /identity/v3/users/a4c718641b5e42f6ba3e7d5819367430 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:10.458754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d33adf-4e12-48b4-bafe-e194cbce5078 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:10.462481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:10.490897 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe40c-7ef3-4c65-87d1-48337ab79f23 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.531159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d86bdd-ded6-43a8-9479-240ff133fe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.532424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d86bdd-ded6-43a8-9479-240ff133fe25 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.532911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d86bdd-ded6-43a8-9479-240ff133fe25 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.533456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d86bdd-ded6-43a8-9479-240ff133fe25 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e/users/a4c718641b5e42f6ba3e7d5819367430/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.535825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d86bdd-ded6-43a8-9479-240ff133fe25 None admin] RBAC: Authorizing `identity:create_grant(project_id=e44154970e4b4efda8f5cd260edbc63e, user_id=a4c718641b5e42f6ba3e7d5819367430, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.537697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d86bdd-ded6-43a8-9479-240ff133fe25 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.553900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 116/231] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:10 2022] PUT /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e/users/a4c718641b5e42f6ba3e7d5819367430/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:10.555401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d86bdd-ded6-43a8-9479-240ff133fe25 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:10.581814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d33adf-4e12-48b4-bafe-e194cbce5078 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.648145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.648917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.649263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.649647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.650584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] RBAC: Adding query filter params (role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=e44154970e4b4efda8f5cd260edbc63e, user.id=b84a8f049f88417596e197dd574ac961) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:10.650963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.652543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.665276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 116/232] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:10 2022] GET /identity/v3/role_assignments?user.id=b84a8f049f88417596e197dd574ac961&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:10.675381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d86bdd-ded6-43a8-9479-240ff133fe25 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.700667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:10.718759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3226a-9ff4-41fe-8e2c-19f39598bdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.720013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3226a-9ff4-41fe-8e2c-19f39598bdfe None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.720546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3226a-9ff4-41fe-8e2c-19f39598bdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.721010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3226a-9ff4-41fe-8e2c-19f39598bdfe None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.722535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3226a-9ff4-41fe-8e2c-19f39598bdfe None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.724326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3226a-9ff4-41fe-8e2c-19f39598bdfe None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.726786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 117/233] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:10 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:10.732877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:10.838813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:10.880196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb26ca-c630-4515-b49a-122c1ce360df None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:10.920423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4541a-231a-4bd6-a60e-5f8e7abe9b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:10.921331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4541a-231a-4bd6-a60e-5f8e7abe9b26 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:10.921611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4541a-231a-4bd6-a60e-5f8e7abe9b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:10.921876 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4541a-231a-4bd6-a60e-5f8e7abe9b26 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e/users/a4c718641b5e42f6ba3e7d5819367430/roles/798b7bcb53684ab49ae899f8902f444a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:10.924163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4541a-231a-4bd6-a60e-5f8e7abe9b26 None admin] RBAC: Authorizing `identity:create_grant(project_id=e44154970e4b4efda8f5cd260edbc63e, user_id=a4c718641b5e42f6ba3e7d5819367430, role_id=798b7bcb53684ab49ae899f8902f444a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:10.925640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4541a-231a-4bd6-a60e-5f8e7abe9b26 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:10.942432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 117/234] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:10 2022] PUT /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e/users/a4c718641b5e42f6ba3e7d5819367430/roles/798b7bcb53684ab49ae899f8902f444a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:11.787593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefb20f-ec43-4d57-8fd8-aba7d09f47dc None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:11.788253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefb20f-ec43-4d57-8fd8-aba7d09f47dc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:11.788253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefb20f-ec43-4d57-8fd8-aba7d09f47dc None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:11.789750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 118/235] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:11.795061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038623a-446d-4021-851e-bde18a2886f6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:11.795449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038623a-446d-4021-851e-bde18a2886f6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:11.795794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038623a-446d-4021-851e-bde18a2886f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:11.817128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c038623a-446d-4021-851e-bde18a2886f6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:11.818462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c038623a-446d-4021-851e-bde18a2886f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:11.903036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 118/236] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:11.934534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefb20f-ec43-4d57-8fd8-aba7d09f47dc None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:12.052327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefb20f-ec43-4d57-8fd8-aba7d09f47dc None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:12.055382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89cb6-8791-46f6-a998-136956c991bd None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:12.055840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89cb6-8791-46f6-a998-136956c991bd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:12.055914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89cb6-8791-46f6-a998-136956c991bd None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:12.058926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 119/237] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:12.064844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b82be9-c389-4e40-98e9-dbba89ceb495 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:12.065130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b82be9-c389-4e40-98e9-dbba89ceb495 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:12.065403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b82be9-c389-4e40-98e9-dbba89ceb495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:12.086425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-28b82be9-c389-4e40-98e9-dbba89ceb495 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:12.087888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-28b82be9-c389-4e40-98e9-dbba89ceb495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:12.142740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f988f9-4da5-46d6-9952-45c876cee013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:12.144005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f988f9-4da5-46d6-9952-45c876cee013 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:12.144481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f988f9-4da5-46d6-9952-45c876cee013 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:12.144941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f988f9-4da5-46d6-9952-45c876cee013 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:12.166663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f988f9-4da5-46d6-9952-45c876cee013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:12.175315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-11f988f9-4da5-46d6-9952-45c876cee013 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:12.182958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:12.183928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:12.184722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:12.188240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 119/238] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:12.199320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f988f9-4da5-46d6-9952-45c876cee013 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:12.200933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 120/239] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:12.225885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b82be9-c389-4e40-98e9-dbba89ceb495 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:12.280672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:12.376316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f988f9-4da5-46d6-9952-45c876cee013 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:12.379805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b82be9-c389-4e40-98e9-dbba89ceb495 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:12.440508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:12.441190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:12.441471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:12.441747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:12.442505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:12.442814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:12.444351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:12.447595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc25cf-6cbc-4f16-928f-8dd6342f9790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:12.448403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc25cf-6cbc-4f16-928f-8dd6342f9790 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:12.448682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc25cf-6cbc-4f16-928f-8dd6342f9790 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:12.448955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc25cf-6cbc-4f16-928f-8dd6342f9790 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:12.450847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc25cf-6cbc-4f16-928f-8dd6342f9790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:12.452347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc25cf-6cbc-4f16-928f-8dd6342f9790 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:12.453982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 121/240] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:12.462347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc25cf-6cbc-4f16-928f-8dd6342f9790 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:12.473450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 120/241] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:12.480061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:12.538360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:12.590180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc25cf-6cbc-4f16-928f-8dd6342f9790 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:12.657377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4738e0-3604-4827-93dc-dd9c88502c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:12.657982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4738e0-3604-4827-93dc-dd9c88502c41 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:12.658262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4738e0-3604-4827-93dc-dd9c88502c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:12.658534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4738e0-3604-4827-93dc-dd9c88502c41 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:12.659284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4738e0-3604-4827-93dc-dd9c88502c41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:12.660768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4738e0-3604-4827-93dc-dd9c88502c41 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:12.694188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 122/242] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:12 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:12.704954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aca2b-07f9-4c2b-a125-19256d6f3b2f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:12.723647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:12.759555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d821d8b-a2d3-4b9e-92dc-3555cf3deec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:12.760479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d821d8b-a2d3-4b9e-92dc-3555cf3deec3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:12.760763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d821d8b-a2d3-4b9e-92dc-3555cf3deec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:12.761042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d821d8b-a2d3-4b9e-92dc-3555cf3deec3 None admin] PATH_INFO: `/v3/users/ed0adb9559924bfb8c7e9a6647a36cca` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:12.785718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d821d8b-a2d3-4b9e-92dc-3555cf3deec3 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0adb9559924bfb8c7e9a6647a36cca)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:12.787370 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d821d8b-a2d3-4b9e-92dc-3555cf3deec3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:12.789559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 121/243] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:12 2022] GET /identity/v3/users/ed0adb9559924bfb8c7e9a6647a36cca => generated 297 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:12.795644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4738e0-3604-4827-93dc-dd9c88502c41 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:12.973901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4738e0-3604-4827-93dc-dd9c88502c41 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:12.994461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:13.045614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6973191-30e8-416d-b23d-5bea43105f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:13.046291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6973191-30e8-416d-b23d-5bea43105f04 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.046580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6973191-30e8-416d-b23d-5bea43105f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.046859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6973191-30e8-416d-b23d-5bea43105f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.138816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6973191-30e8-416d-b23d-5bea43105f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:13.140388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6973191-30e8-416d-b23d-5bea43105f04 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:13.152940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade5918-cba5-45b3-bb62-cfa655cdbc75 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.153233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade5918-cba5-45b3-bb62-cfa655cdbc75 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.153491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade5918-cba5-45b3-bb62-cfa655cdbc75 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.154927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 122/244] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:13.161553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0d2e4-a81d-461e-a2f1-28e51e871a43 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.161837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0d2e4-a81d-461e-a2f1-28e51e871a43 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.162377 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0d2e4-a81d-461e-a2f1-28e51e871a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.185716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c2d0d2e4-a81d-461e-a2f1-28e51e871a43 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:13.187025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c2d0d2e4-a81d-461e-a2f1-28e51e871a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:13.257889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 123/245] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:13.259169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f9316-8d30-4b4c-bf76-1098c184fdce None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.260400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f9316-8d30-4b4c-bf76-1098c184fdce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.260668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f9316-8d30-4b4c-bf76-1098c184fdce None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.262233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 124/246] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:13.265307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f9316-8d30-4b4c-bf76-1098c184fdce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:13.287862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 123/247] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:13.289337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e19da-6118-4ea8-9930-5b4aa30c39f6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.289602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e19da-6118-4ea8-9930-5b4aa30c39f6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.289889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e19da-6118-4ea8-9930-5b4aa30c39f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.290243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:13.307515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4d8e19da-6118-4ea8-9930-5b4aa30c39f6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:13.308794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4d8e19da-6118-4ea8-9930-5b4aa30c39f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:13.392605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f9316-8d30-4b4c-bf76-1098c184fdce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:13.413707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 124/248] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:13.415004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e19da-6118-4ea8-9930-5b4aa30c39f6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:13.460134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59680978-6fcb-42ad-aa21-695f6c42f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:13.462570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680978-6fcb-42ad-aa21-695f6c42f972 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.463560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680978-6fcb-42ad-aa21-695f6c42f972 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.465456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680978-6fcb-42ad-aa21-695f6c42f972 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.467879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59680978-6fcb-42ad-aa21-695f6c42f972 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:13.469686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59680978-6fcb-42ad-aa21-695f6c42f972 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:13.477473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:13.477638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 125/249] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:13 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:13.478504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59680978-6fcb-42ad-aa21-695f6c42f972 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:13.572383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e19da-6118-4ea8-9930-5b4aa30c39f6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:13.584202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:13.605015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59680978-6fcb-42ad-aa21-695f6c42f972 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:13.612577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d22c0-b1dd-4784-94cb-a6a27f2b22ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:13.613391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d22c0-b1dd-4784-94cb-a6a27f2b22ca None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.613691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d22c0-b1dd-4784-94cb-a6a27f2b22ca None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.613951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d22c0-b1dd-4784-94cb-a6a27f2b22ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.625228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d22c0-b1dd-4784-94cb-a6a27f2b22ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-486d22c0-b1dd-4784-94cb-a6a27f2b22ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:13.633359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:13.634278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:13.635073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 125/250] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:13.635775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d22c0-b1dd-4784-94cb-a6a27f2b22ca None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:13.648251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d3fb1-0261-475c-96cd-8797a2bb89aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:13.649221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d3fb1-0261-475c-96cd-8797a2bb89aa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.649593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d3fb1-0261-475c-96cd-8797a2bb89aa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.649924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d3fb1-0261-475c-96cd-8797a2bb89aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.661321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d3fb1-0261-475c-96cd-8797a2bb89aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-642d3fb1-0261-475c-96cd-8797a2bb89aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:13.669525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:13.671918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 126/251] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:13.674796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba0eb-1004-40df-a41e-4c2781f5a259 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.679939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba0eb-1004-40df-a41e-4c2781f5a259 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.679939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba0eb-1004-40df-a41e-4c2781f5a259 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.679939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 127/252] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:13.679939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ba0eb-1004-40df-a41e-4c2781f5a259 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:13.739374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:13.759530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d22c0-b1dd-4784-94cb-a6a27f2b22ca None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:13.786417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ba0eb-1004-40df-a41e-4c2781f5a259 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:13.804951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:13.806302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.807348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.807660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.808514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] RBAC: Adding query filter params (role.id=e7d8e4fc6f35425ab4e5c87ed577de1f, scope.project.id=e44154970e4b4efda8f5cd260edbc63e, user.id=ed0adb9559924bfb8c7e9a6647a36cca) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:13.808824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:13.810275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:13.816404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 126/253] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:13 2022] GET /identity/v3/role_assignments?user.id=ed0adb9559924bfb8c7e9a6647a36cca&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=e7d8e4fc6f35425ab4e5c87ed577de1f => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:13.817304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:13.827081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79834-ad14-4be1-bcfd-4cdd319958a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:13.828045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79834-ad14-4be1-bcfd-4cdd319958a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.828467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79834-ad14-4be1-bcfd-4cdd319958a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.828908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79834-ad14-4be1-bcfd-4cdd319958a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.829922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79834-ad14-4be1-bcfd-4cdd319958a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:13.830382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79834-ad14-4be1-bcfd-4cdd319958a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:13.832185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79834-ad14-4be1-bcfd-4cdd319958a2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:13.842357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 128/254] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:13.848696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ff0a-c172-4df2-bb49-05f1f1ed5ab7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:13.849322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ff0a-c172-4df2-bb49-05f1f1ed5ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:13.849828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ff0a-c172-4df2-bb49-05f1f1ed5ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:13.855364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:13.869104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c638ff0a-c172-4df2-bb49-05f1f1ed5ab7 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:13.870735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c638ff0a-c172-4df2-bb49-05f1f1ed5ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:13.948126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 129/255] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:13.949389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638ff0a-c172-4df2-bb49-05f1f1ed5ab7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:14.039357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f79bfe-07b9-4945-8904-9fa4b64da74a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:14.066485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638ff0a-c172-4df2-bb49-05f1f1ed5ab7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:14.099759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:14.108410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf379fa-ce39-4326-85ed-6ee4526d15f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:14.109578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf379fa-ce39-4326-85ed-6ee4526d15f1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:14.110540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf379fa-ce39-4326-85ed-6ee4526d15f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:14.111624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf379fa-ce39-4326-85ed-6ee4526d15f1 None admin] PATH_INFO: `/v3/users/a4c718641b5e42f6ba3e7d5819367430` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:14.113011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf379fa-ce39-4326-85ed-6ee4526d15f1 None admin] RBAC: Authorizing `identity:get_user(user_id=a4c718641b5e42f6ba3e7d5819367430)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:14.114748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:14.115493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf379fa-ce39-4326-85ed-6ee4526d15f1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:14.115938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:14.116415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:14.116884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:14.117208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 130/256] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:13 2022] GET /identity/v3/users/a4c718641b5e42f6ba3e7d5819367430 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:14.117987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf379fa-ce39-4326-85ed-6ee4526d15f1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:14.118277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:14.123230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:14.125037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:14.135340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 127/257] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:14.136424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:14.157647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:14.247799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0b9e3-ca75-4b44-9a6b-b5c137c5dae7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:14.283741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5990d-457c-44cd-b982-f19d7f477ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:14.284872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5990d-457c-44cd-b982-f19d7f477ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:14.285373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5990d-457c-44cd-b982-f19d7f477ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:14.285844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5990d-457c-44cd-b982-f19d7f477ce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:14.368168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5990d-457c-44cd-b982-f19d7f477ce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:14.369997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5990d-457c-44cd-b982-f19d7f477ce1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:14.387643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:14.425182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf379fa-ce39-4326-85ed-6ee4526d15f1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:14.476301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 128/258] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:14.477475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5990d-457c-44cd-b982-f19d7f477ce1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:14.499935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925955d-1071-437d-80aa-060b0dcf9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:14.500768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925955d-1071-437d-80aa-060b0dcf9550 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:14.501044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925955d-1071-437d-80aa-060b0dcf9550 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:14.501314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925955d-1071-437d-80aa-060b0dcf9550 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:14.523160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925955d-1071-437d-80aa-060b0dcf9550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c925955d-1071-437d-80aa-060b0dcf9550 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:14.542918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:14.543978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:14.544943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 131/259] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:14.556187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925955d-1071-437d-80aa-060b0dcf9550 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:14.596016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5990d-457c-44cd-b982-f19d7f477ce1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:14.611846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:14.634253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961db6d-30d7-4f52-88e2-cf3d1d03fe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:14.635656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961db6d-30d7-4f52-88e2-cf3d1d03fe0b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:14.636079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961db6d-30d7-4f52-88e2-cf3d1d03fe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:14.636494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961db6d-30d7-4f52-88e2-cf3d1d03fe0b None admin] PATH_INFO: `/v3/users/b84a8f049f88417596e197dd574ac961` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:14.637932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961db6d-30d7-4f52-88e2-cf3d1d03fe0b None admin] RBAC: Authorizing `identity:get_user(user_id=b84a8f049f88417596e197dd574ac961)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:14.639799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961db6d-30d7-4f52-88e2-cf3d1d03fe0b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:14.641653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 129/260] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:14 2022] GET /identity/v3/users/b84a8f049f88417596e197dd574ac961 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:14.642548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961db6d-30d7-4f52-88e2-cf3d1d03fe0b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:14.710998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:14.782567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925955d-1071-437d-80aa-060b0dcf9550 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:14.785987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961db6d-30d7-4f52-88e2-cf3d1d03fe0b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:14.844051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:14.844738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:14.845025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:14.845299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:14.846066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:14.846377 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:14.847802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:14.853117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d92543-7041-4165-9369-36ffbd0c5263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:14.854007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92543-7041-4165-9369-36ffbd0c5263 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:14.854292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92543-7041-4165-9369-36ffbd0c5263 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:14.854573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92543-7041-4165-9369-36ffbd0c5263 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:14.855793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d92543-7041-4165-9369-36ffbd0c5263 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:14.857273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d92543-7041-4165-9369-36ffbd0c5263 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:14.859927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 132/261] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:14 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:14.860693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d92543-7041-4165-9369-36ffbd0c5263 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:14.869551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 130/262] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:14.870380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:14.967692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:14.980976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d92543-7041-4165-9369-36ffbd0c5263 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:15.010178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:15.019012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0b70e-68fd-4e39-85b7-ee9d5eb8e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:15.019924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0b70e-68fd-4e39-85b7-ee9d5eb8e037 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.020207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0b70e-68fd-4e39-85b7-ee9d5eb8e037 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.020479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0b70e-68fd-4e39-85b7-ee9d5eb8e037 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.022407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0b70e-68fd-4e39-85b7-ee9d5eb8e037 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:15.023947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0b70e-68fd-4e39-85b7-ee9d5eb8e037 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:15.025696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 133/263] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:14 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:15.026470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0b70e-68fd-4e39-85b7-ee9d5eb8e037 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:15.038372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f2768-6963-4ce6-ba7b-82756c1b8570 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:15.072014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:15.072674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.072953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.073226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.074068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=e44154970e4b4efda8f5cd260edbc63e, user.id=a4c718641b5e42f6ba3e7d5819367430) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:15.074378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:15.075861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:15.082275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 131/264] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:14 2022] GET /identity/v3/role_assignments?user.id=a4c718641b5e42f6ba3e7d5819367430&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:15.083151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:15.202988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0b70e-68fd-4e39-85b7-ee9d5eb8e037 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:15.275734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b449f-1b08-4fed-985b-0d9a6ba947db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:15.276552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b449f-1b08-4fed-985b-0d9a6ba947db None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.276833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b6675-82ea-4769-92fc-ab5a77f323f1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:15.277007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b449f-1b08-4fed-985b-0d9a6ba947db None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.277153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b449f-1b08-4fed-985b-0d9a6ba947db None admin] PATH_INFO: `/v3/users/a4c718641b5e42f6ba3e7d5819367430` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.284175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b449f-1b08-4fed-985b-0d9a6ba947db None admin] RBAC: Authorizing `identity:get_user(user_id=a4c718641b5e42f6ba3e7d5819367430)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:15.285854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b449f-1b08-4fed-985b-0d9a6ba947db None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:15.286884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:15.289253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 134/265] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:15 2022] GET /identity/v3/users/a4c718641b5e42f6ba3e7d5819367430 => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:15.296166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b449f-1b08-4fed-985b-0d9a6ba947db None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:15.344828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b470b4-dc9c-4bfa-9f96-4d09ba7d1897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:15.345745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b470b4-dc9c-4bfa-9f96-4d09ba7d1897 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.346020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b470b4-dc9c-4bfa-9f96-4d09ba7d1897 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.346290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b470b4-dc9c-4bfa-9f96-4d09ba7d1897 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e/users/b84a8f049f88417596e197dd574ac961/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.352628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b470b4-dc9c-4bfa-9f96-4d09ba7d1897 None admin] RBAC: Authorizing `identity:create_grant(project_id=e44154970e4b4efda8f5cd260edbc63e, user_id=b84a8f049f88417596e197dd574ac961, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:15.354117 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b470b4-dc9c-4bfa-9f96-4d09ba7d1897 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:15.366307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:15.385535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 132/266] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:15 2022] PUT /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e/users/b84a8f049f88417596e197dd574ac961/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:15.420152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b449f-1b08-4fed-985b-0d9a6ba947db None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:15.462305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db7e85-2123-4975-801d-033db11b1841 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.462589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db7e85-2123-4975-801d-033db11b1841 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.462880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db7e85-2123-4975-801d-033db11b1841 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.465228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 133/267] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:15.469010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a8f9e-3369-49ef-b83f-ed000434f4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:15.469687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8f9e-3369-49ef-b83f-ed000434f4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.469976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8f9e-3369-49ef-b83f-ed000434f4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.470301 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8f9e-3369-49ef-b83f-ed000434f4d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.470929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abae70-3475-4ca8-85bd-8d3dddfde791 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.471241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abae70-3475-4ca8-85bd-8d3dddfde791 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.471537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abae70-3475-4ca8-85bd-8d3dddfde791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.520349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e0abae70-3475-4ca8-85bd-8d3dddfde791 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:15.521620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e0abae70-3475-4ca8-85bd-8d3dddfde791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:15.568338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a8f9e-3369-49ef-b83f-ed000434f4d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:15.570431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a8f9e-3369-49ef-b83f-ed000434f4d7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:15.639069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 134/268] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:15.650270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:15.680593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abae70-3475-4ca8-85bd-8d3dddfde791 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:15.700115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 135/269] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:15.706412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a8f9e-3369-49ef-b83f-ed000434f4d7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:15.857301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:15.866176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abae70-3475-4ca8-85bd-8d3dddfde791 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:15.873706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a8f9e-3369-49ef-b83f-ed000434f4d7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:15.912378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7e6f2-6eaf-4419-83c1-7f970ffaf8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:15.913796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7e6f2-6eaf-4419-83c1-7f970ffaf8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.914200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7e6f2-6eaf-4419-83c1-7f970ffaf8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.914487 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7e6f2-6eaf-4419-83c1-7f970ffaf8f5 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.916357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7e6f2-6eaf-4419-83c1-7f970ffaf8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:15.918348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7e6f2-6eaf-4419-83c1-7f970ffaf8f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:15.921143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 136/270] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:15 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:15.923737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337093f3-4051-4743-9cef-cb4a046636d9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.924089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337093f3-4051-4743-9cef-cb4a046636d9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.924439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337093f3-4051-4743-9cef-cb4a046636d9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.926034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 137/271] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:15.930891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337093f3-4051-4743-9cef-cb4a046636d9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:15.931732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede946d-cd4f-4a59-ab2b-4a0bf7e3a833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:15.932618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede946d-cd4f-4a59-ab2b-4a0bf7e3a833 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.932974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede946d-cd4f-4a59-ab2b-4a0bf7e3a833 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.933313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede946d-cd4f-4a59-ab2b-4a0bf7e3a833 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.935604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede946d-cd4f-4a59-ab2b-4a0bf7e3a833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:15.937082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede946d-cd4f-4a59-ab2b-4a0bf7e3a833 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:15.939025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 135/272] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:15.940233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c216-ef0a-40d7-ba72-ff0e25904851 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:15.940517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c216-ef0a-40d7-ba72-ff0e25904851 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:15.940793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c216-ef0a-40d7-ba72-ff0e25904851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:15.961540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-14d9c216-ef0a-40d7-ba72-ff0e25904851 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:15.963895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-14d9c216-ef0a-40d7-ba72-ff0e25904851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:15.969491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:16.038949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 136/273] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:16.040112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c216-ef0a-40d7-ba72-ff0e25904851 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:16.059065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337093f3-4051-4743-9cef-cb4a046636d9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:16.102709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:16.103759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.103897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.104206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.105256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] RBAC: Adding query filter params (role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=e44154970e4b4efda8f5cd260edbc63e, user.id=a4c718641b5e42f6ba3e7d5819367430) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:16.105642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:16.107716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:16.114905 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 138/274] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:15 2022] GET /identity/v3/role_assignments?user.id=a4c718641b5e42f6ba3e7d5819367430&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=798b7bcb53684ab49ae899f8902f444a => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:16.123234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:16.189339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:16.198686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c216-ef0a-40d7-ba72-ff0e25904851 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:16.280989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829477e-a792-4a5e-bc18-00ec606ec0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:16.281686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829477e-a792-4a5e-bc18-00ec606ec0a3 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.281971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829477e-a792-4a5e-bc18-00ec606ec0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.282249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829477e-a792-4a5e-bc18-00ec606ec0a3 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.282987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829477e-a792-4a5e-bc18-00ec606ec0a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:16.284480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829477e-a792-4a5e-bc18-00ec606ec0a3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:16.289990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:16.295834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64babda6-49bd-4de5-b40a-9977c56752fa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:16.315666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 137/275] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:16 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:16.341353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20985dd-9ceb-4eb0-a2c1-d6a8e9f989e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:16.342504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20985dd-9ceb-4eb0-a2c1-d6a8e9f989e6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.343033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20985dd-9ceb-4eb0-a2c1-d6a8e9f989e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.343555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20985dd-9ceb-4eb0-a2c1-d6a8e9f989e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.356316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20985dd-9ceb-4eb0-a2c1-d6a8e9f989e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f20985dd-9ceb-4eb0-a2c1-d6a8e9f989e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:16.365070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:16.366178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:16.367154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:16.369734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 139/276] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:16.376112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829477e-a792-4a5e-bc18-00ec606ec0a3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:16.493816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829477e-a792-4a5e-bc18-00ec606ec0a3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:16.510192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec84ac-e5c2-46a7-83ab-725a627de53f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.510592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec84ac-e5c2-46a7-83ab-725a627de53f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.510774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec84ac-e5c2-46a7-83ab-725a627de53f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.512330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 140/277] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:16.517697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f4623-c8ab-4326-b30c-838f71ca27bb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.518450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f4623-c8ab-4326-b30c-838f71ca27bb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.519344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f4623-c8ab-4326-b30c-838f71ca27bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.540747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:16.540747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.540747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.541057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.541985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:16.541985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:16.543376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:16.551916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cd3f4623-c8ab-4326-b30c-838f71ca27bb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:16.553197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cd3f4623-c8ab-4326-b30c-838f71ca27bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:16.554692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 138/278] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:16.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:16.579295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:16.640760 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 141/279] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:16.665130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f4623-c8ab-4326-b30c-838f71ca27bb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:16.693003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f5714-b9ad-4228-a398-facf011472ec None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:16.735804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66149a61-7eeb-49f7-962c-4cd7b0a19a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:16.736701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149a61-7eeb-49f7-962c-4cd7b0a19a94 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.736979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149a61-7eeb-49f7-962c-4cd7b0a19a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.737251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149a61-7eeb-49f7-962c-4cd7b0a19a94 None admin] PATH_INFO: `/v3/users/aa009f1ee665404b9295a09fd76e19b8` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.751790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66149a61-7eeb-49f7-962c-4cd7b0a19a94 None admin] RBAC: Authorizing `identity:get_user(user_id=aa009f1ee665404b9295a09fd76e19b8)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:16.753279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66149a61-7eeb-49f7-962c-4cd7b0a19a94 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:16.755052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 139/280] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:16 2022] GET /identity/v3/users/aa009f1ee665404b9295a09fd76e19b8 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:16.760877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66149a61-7eeb-49f7-962c-4cd7b0a19a94 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:16.768666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:16.785211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f4623-c8ab-4326-b30c-838f71ca27bb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:16.831958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9cee1-9fe6-475b-a5a9-c77c7fb17077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:16.832766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9cee1-9fe6-475b-a5a9-c77c7fb17077 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.833047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9cee1-9fe6-475b-a5a9-c77c7fb17077 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.833317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9cee1-9fe6-475b-a5a9-c77c7fb17077 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.847083 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9cee1-9fe6-475b-a5a9-c77c7fb17077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b7e9cee1-9fe6-475b-a5a9-c77c7fb17077 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:16.857137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:16.858058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:16.858847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:16.859492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:16.859896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 142/281] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:16.864969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9cee1-9fe6-475b-a5a9-c77c7fb17077 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:16.893782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66149a61-7eeb-49f7-962c-4cd7b0a19a94 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:16.935263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:16.935932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:16.936211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:16.936711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:16.937562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] RBAC: Adding query filter params (role.id=af1e89bc75e74c9f819959d898d086a5, user.id=aa009f1ee665404b9295a09fd76e19b8, scope.system=all) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:16.937873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:16.939366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:16.951571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 140/282] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 23 08:07:16 2022] GET /identity/v3/role_assignments?user.id=aa009f1ee665404b9295a09fd76e19b8&scope.system=all&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:17.039146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9cee1-9fe6-475b-a5a9-c77c7fb17077 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:17.087033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:17.087748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:17.088026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:17.088298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:17.089064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:17.089367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:17.090839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:17.103668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 143/283] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:17.109104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:17.243601 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304adc21-eecf-44b7-8592-1c480d91ac11 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:17.289440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6663b-88f5-456b-9037-d8b4544cfb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:17.290326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6663b-88f5-456b-9037-d8b4544cfb94 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:17.290608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6663b-88f5-456b-9037-d8b4544cfb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:17.290879 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6663b-88f5-456b-9037-d8b4544cfb94 None admin] PATH_INFO: `/v3/users/ed0adb9559924bfb8c7e9a6647a36cca` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:17.292090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6663b-88f5-456b-9037-d8b4544cfb94 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0adb9559924bfb8c7e9a6647a36cca)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:17.293533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6663b-88f5-456b-9037-d8b4544cfb94 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:17.295557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 141/284] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:17 2022] GET /identity/v3/users/ed0adb9559924bfb8c7e9a6647a36cca => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:17.300633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:17.313094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:17.405002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d29db5-2e8d-48be-8400-a6af1f792f1c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:17.439435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956d904-4a1c-4954-9fd8-a2c15fc92a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:17.440656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d904-4a1c-4954-9fd8-a2c15fc92a9c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:17.441105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d904-4a1c-4954-9fd8-a2c15fc92a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:17.441543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d904-4a1c-4954-9fd8-a2c15fc92a9c None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:17.442906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956d904-4a1c-4954-9fd8-a2c15fc92a9c None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:17.444694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956d904-4a1c-4954-9fd8-a2c15fc92a9c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:17.446965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 144/285] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:17 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:17.452155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6663b-88f5-456b-9037-d8b4544cfb94 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:17.460514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:17.587485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6663b-88f5-456b-9037-d8b4544cfb94 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:17.634243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24850495-7957-4c9e-b22c-8145c6af9793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:17.635143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24850495-7957-4c9e-b22c-8145c6af9793 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:17.635448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24850495-7957-4c9e-b22c-8145c6af9793 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:17.635721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24850495-7957-4c9e-b22c-8145c6af9793 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e/users/ed0adb9559924bfb8c7e9a6647a36cca/roles/e7d8e4fc6f35425ab4e5c87ed577de1f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:17.639974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24850495-7957-4c9e-b22c-8145c6af9793 None admin] RBAC: Authorizing `identity:create_grant(project_id=e44154970e4b4efda8f5cd260edbc63e, user_id=ed0adb9559924bfb8c7e9a6647a36cca, role_id=e7d8e4fc6f35425ab4e5c87ed577de1f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:17.641423 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24850495-7957-4c9e-b22c-8145c6af9793 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:17.660855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 142/286] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:17 2022] PUT /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e/users/ed0adb9559924bfb8c7e9a6647a36cca/roles/e7d8e4fc6f35425ab4e5c87ed577de1f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:17.713981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67fa52-c827-4720-abea-4af3f2e9d46a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:17.714507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67fa52-c827-4720-abea-4af3f2e9d46a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:17.714507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67fa52-c827-4720-abea-4af3f2e9d46a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:17.716450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 145/287] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:17.722286 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a72de-abf0-4b7e-b812-6b45a81961b6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:17.722600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a72de-abf0-4b7e-b812-6b45a81961b6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:17.722893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a72de-abf0-4b7e-b812-6b45a81961b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:17.744617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1f9a72de-abf0-4b7e-b812-6b45a81961b6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:17.745902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1f9a72de-abf0-4b7e-b812-6b45a81961b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:17.835307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:17.844526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 143/288] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:17.869111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67fa52-c827-4720-abea-4af3f2e9d46a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:17.972687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:18.012837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67fa52-c827-4720-abea-4af3f2e9d46a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:18.056944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5c630-fc2e-4611-b635-2de9d21b0a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:18.057758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c630-fc2e-4611-b635-2de9d21b0a0b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:18.058041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c630-fc2e-4611-b635-2de9d21b0a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:18.058311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c630-fc2e-4611-b635-2de9d21b0a0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:18.072365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5c630-fc2e-4611-b635-2de9d21b0a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-abd5c630-fc2e-4611-b635-2de9d21b0a0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:18.080229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:18.081116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:18.081901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:18.086266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 146/289] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:18.087564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a72de-abf0-4b7e-b812-6b45a81961b6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:18.254973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a72de-abf0-4b7e-b812-6b45a81961b6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:18.303659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:18.305233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:18.305711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:18.306178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:18.307138 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:18.307658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:18.309252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:18.323243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 144/290] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:18.328739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5c630-fc2e-4611-b635-2de9d21b0a0b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:18.479276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5c630-fc2e-4611-b635-2de9d21b0a0b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:18.536133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556dd1b-ce4f-4db5-b00f-8f6e76c89008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:18.537058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556dd1b-ce4f-4db5-b00f-8f6e76c89008 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:18.537421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556dd1b-ce4f-4db5-b00f-8f6e76c89008 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:18.537700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556dd1b-ce4f-4db5-b00f-8f6e76c89008 None admin] PATH_INFO: `/v3/users/b84a8f049f88417596e197dd574ac961` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:18.541759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556dd1b-ce4f-4db5-b00f-8f6e76c89008 None admin] RBAC: Authorizing `identity:get_user(user_id=b84a8f049f88417596e197dd574ac961)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:18.543296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556dd1b-ce4f-4db5-b00f-8f6e76c89008 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:18.545016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 147/291] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:18 2022] GET /identity/v3/users/b84a8f049f88417596e197dd574ac961 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:18.551779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:18.634325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:18.690209 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c925e0-c6c8-41b7-b2ba-36f37fb47bc6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:18.727585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac410c5-d57b-4568-bc3b-ca35195117a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:18.728243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac410c5-d57b-4568-bc3b-ca35195117a2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:18.728524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac410c5-d57b-4568-bc3b-ca35195117a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:18.728790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac410c5-d57b-4568-bc3b-ca35195117a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:18.819253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac410c5-d57b-4568-bc3b-ca35195117a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:18.820660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac410c5-d57b-4568-bc3b-ca35195117a2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:18.890425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:18.943025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 145/292] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 08:07:18.948716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556dd1b-ce4f-4db5-b00f-8f6e76c89008 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:18.952551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:19.043132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf36b89-9fb9-4709-889f-844dcbb91451 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.043503 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf36b89-9fb9-4709-889f-844dcbb91451 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.043734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf36b89-9fb9-4709-889f-844dcbb91451 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.045367 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 146/293] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:19.050199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb8a4-750b-4e81-aaee-9ddffc2ce88c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.050643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb8a4-750b-4e81-aaee-9ddffc2ce88c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.050918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb8a4-750b-4e81-aaee-9ddffc2ce88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.073064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4dffb8a4-750b-4e81-aaee-9ddffc2ce88c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:19.074464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4dffb8a4-750b-4e81-aaee-9ddffc2ce88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:19.101442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556dd1b-ce4f-4db5-b00f-8f6e76c89008 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:19.173926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f3be8-4c38-4884-b52e-8b3b4b857240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:19.175257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f3be8-4c38-4884-b52e-8b3b4b857240 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.177104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f3be8-4c38-4884-b52e-8b3b4b857240 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.177654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f3be8-4c38-4884-b52e-8b3b4b857240 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.181088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f3be8-4c38-4884-b52e-8b3b4b857240 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:19.186146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f3be8-4c38-4884-b52e-8b3b4b857240 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:19.190864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 148/294] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:18 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:19.196641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 147/295] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:19.198196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f3be8-4c38-4884-b52e-8b3b4b857240 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:19.220470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffb8a4-750b-4e81-aaee-9ddffc2ce88c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:19.326922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:19.348688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:19.360540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f3be8-4c38-4884-b52e-8b3b4b857240 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:19.365389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffb8a4-750b-4e81-aaee-9ddffc2ce88c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:19.394997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3743-a8c9-4a85-b452-e0eb33ece073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:19.395702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3743-a8c9-4a85-b452-e0eb33ece073 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.395988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3743-a8c9-4a85-b452-e0eb33ece073 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.396306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3743-a8c9-4a85-b452-e0eb33ece073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.397147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a3743-a8c9-4a85-b452-e0eb33ece073 None admin] RBAC: Adding query filter params (role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=e44154970e4b4efda8f5cd260edbc63e, user.id=b84a8f049f88417596e197dd574ac961) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:19.397468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a3743-a8c9-4a85-b452-e0eb33ece073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:19.398931 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a3743-a8c9-4a85-b452-e0eb33ece073 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:19.402280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e7987-0646-408c-831e-537ff67c67f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:19.403359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e7987-0646-408c-831e-537ff67c67f5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.403829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e7987-0646-408c-831e-537ff67c67f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.404268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e7987-0646-408c-831e-537ff67c67f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.409586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 149/296] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:19 2022] GET /identity/v3/role_assignments?user.id=b84a8f049f88417596e197dd574ac961&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:19.409586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c27bc-b5d1-4d5c-9b56-fb2c173e198c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.409586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c27bc-b5d1-4d5c-9b56-fb2c173e198c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.409586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c27bc-b5d1-4d5c-9b56-fb2c173e198c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.409586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 150/297] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:19.417244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973489f3-be0c-49fd-88e5-495acde373f1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.417588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973489f3-be0c-49fd-88e5-495acde373f1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.417810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973489f3-be0c-49fd-88e5-495acde373f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.440215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e7987-0646-408c-831e-537ff67c67f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e61e7987-0646-408c-831e-537ff67c67f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:19.453793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:19.454749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:19.455566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:19.456166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-973489f3-be0c-49fd-88e5-495acde373f1 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:19.458474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 148/298] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:19.458584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-973489f3-be0c-49fd-88e5-495acde373f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:19.465865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e7987-0646-408c-831e-537ff67c67f5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:19.584235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 151/299] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:19.585453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbefb1c-ef6a-4fd8-84aa-dc792e83e4ca None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.585843 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbefb1c-ef6a-4fd8-84aa-dc792e83e4ca None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.586116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbefb1c-ef6a-4fd8-84aa-dc792e83e4ca None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.587592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 152/300] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:19.593573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91ee32-0694-4678-b8e5-af9adcd1ad89 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.593840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91ee32-0694-4678-b8e5-af9adcd1ad89 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.594113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91ee32-0694-4678-b8e5-af9adcd1ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.618527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ca91ee32-0694-4678-b8e5-af9adcd1ad89 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:19.620473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ca91ee32-0694-4678-b8e5-af9adcd1ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:19.644261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e7987-0646-408c-831e-537ff67c67f5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:19.682083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:19.683037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.683537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.683950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.684842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:19.685291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:19.686865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:19.696695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 149/301] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:19.697087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:19.697898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:19.713865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 153/302] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:19.715146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91ee32-0694-4678-b8e5-af9adcd1ad89 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:19.718168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:19.819001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3918cf0-1614-4254-a944-e2e328e8afe2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:19.822221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91ee32-0694-4678-b8e5-af9adcd1ad89 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:19.857514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de7127-36e0-4761-b673-c4ba280c9793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:19.858348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e85d4c-5be7-42f0-804a-9adad4eeb113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:19.858658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de7127-36e0-4761-b673-c4ba280c9793 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.858942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de7127-36e0-4761-b673-c4ba280c9793 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.859240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de7127-36e0-4761-b673-c4ba280c9793 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.859337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e85d4c-5be7-42f0-804a-9adad4eeb113 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:19.859520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e85d4c-5be7-42f0-804a-9adad4eeb113 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:19.859818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e85d4c-5be7-42f0-804a-9adad4eeb113 None admin] PATH_INFO: `/v3/users/b599f6ac2b794ca6aa6fc746f7824d60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:19.860648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de7127-36e0-4761-b673-c4ba280c9793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:19.862105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de7127-36e0-4761-b673-c4ba280c9793 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:19.863773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 150/303] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:19.864538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de7127-36e0-4761-b673-c4ba280c9793 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:19.875555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e85d4c-5be7-42f0-804a-9adad4eeb113 None admin] RBAC: Authorizing `identity:get_user(user_id=b599f6ac2b794ca6aa6fc746f7824d60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:19.877249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e85d4c-5be7-42f0-804a-9adad4eeb113 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:19.883935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 154/304] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:19 2022] GET /identity/v3/users/b599f6ac2b794ca6aa6fc746f7824d60 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:19.883935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e85d4c-5be7-42f0-804a-9adad4eeb113 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:19.940326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:19.952006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:19.978722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de7127-36e0-4761-b673-c4ba280c9793 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:19.989283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e85d4c-5be7-42f0-804a-9adad4eeb113 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:20.016254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c3e50-4bb1-4137-bb44-1f9359a53cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.017120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c3e50-4bb1-4137-bb44-1f9359a53cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.017486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c3e50-4bb1-4137-bb44-1f9359a53cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.017773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c3e50-4bb1-4137-bb44-1f9359a53cb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.025358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784b827-3569-4d83-b7cd-3c4a7412387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.026037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b827-3569-4d83-b7cd-3c4a7412387c None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.026317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b827-3569-4d83-b7cd-3c4a7412387c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.026589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b827-3569-4d83-b7cd-3c4a7412387c None admin] PATH_INFO: `/v3/groups` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.027305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784b827-3569-4d83-b7cd-3c4a7412387c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:20.028813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784b827-3569-4d83-b7cd-3c4a7412387c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:20.029449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c3e50-4bb1-4137-bb44-1f9359a53cb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-250c3e50-4bb1-4137-bb44-1f9359a53cb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:20.038081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:20.039061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:20.039878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:20.042035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 151/305] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:20.047818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c3e50-4bb1-4137-bb44-1f9359a53cb6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:20.051415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 155/306] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Jun 23 08:07:19 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:20.052390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784b827-3569-4d83-b7cd-3c4a7412387c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:20.181608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784b827-3569-4d83-b7cd-3c4a7412387c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:20.188629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c3e50-4bb1-4137-bb44-1f9359a53cb6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:20.204387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:20.234442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015e138-96d9-4716-b140-06b149d39b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.235112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015e138-96d9-4716-b140-06b149d39b3c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.235486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015e138-96d9-4716-b140-06b149d39b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.235761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015e138-96d9-4716-b140-06b149d39b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.236604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015e138-96d9-4716-b140-06b149d39b3c None admin] RBAC: Adding query filter params (role.id=e7d8e4fc6f35425ab4e5c87ed577de1f, user.id=b599f6ac2b794ca6aa6fc746f7824d60, scope.system=all) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:20.236921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015e138-96d9-4716-b140-06b149d39b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:20.237210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.238045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.238388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015e138-96d9-4716-b140-06b149d39b3c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:20.238657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.239096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.240089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:20.240539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:20.242050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:20.242385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:20.247332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 152/307] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 23 08:07:20 2022] GET /identity/v3/role_assignments?user.id=b599f6ac2b794ca6aa6fc746f7824d60&scope.system=all&role.id=e7d8e4fc6f35425ab4e5c87ed577de1f => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:20.252904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 156/308] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:20.257693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503a97-c92a-4cc2-9d2c-67233afce33a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.258053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503a97-c92a-4cc2-9d2c-67233afce33a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.258511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503a97-c92a-4cc2-9d2c-67233afce33a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.259505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:20.259984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 153/309] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:20.265666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dfe13-ecfe-4032-90bd-73a0db895fc8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.266955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dfe13-ecfe-4032-90bd-73a0db895fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.266955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dfe13-ecfe-4032-90bd-73a0db895fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.286059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f16dfe13-ecfe-4032-90bd-73a0db895fc8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:20.287834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f16dfe13-ecfe-4032-90bd-73a0db895fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:20.362827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 154/310] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:20.387106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dfe13-ecfe-4032-90bd-73a0db895fc8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:20.391931 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a7025-d595-4667-b2d1-ddd47f1f4b8f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:20.432834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07951c8f-2d1c-42df-9bf2-841bc8623e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.433648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951c8f-2d1c-42df-9bf2-841bc8623e67 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.433931 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951c8f-2d1c-42df-9bf2-841bc8623e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.434204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951c8f-2d1c-42df-9bf2-841bc8623e67 None admin] PATH_INFO: `/v3/users/aa009f1ee665404b9295a09fd76e19b8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.435442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07951c8f-2d1c-42df-9bf2-841bc8623e67 None admin] RBAC: Authorizing `identity:get_user(user_id=aa009f1ee665404b9295a09fd76e19b8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:20.436900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07951c8f-2d1c-42df-9bf2-841bc8623e67 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:20.438841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 157/311] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:20 2022] GET /identity/v3/users/aa009f1ee665404b9295a09fd76e19b8 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:20.448680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07951c8f-2d1c-42df-9bf2-841bc8623e67 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:20.512740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:20.523682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dfe13-ecfe-4032-90bd-73a0db895fc8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:20.557554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:20.565074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312ff66-996f-459f-be56-4a4761afac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.566291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312ff66-996f-459f-be56-4a4761afac68 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.566832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312ff66-996f-459f-be56-4a4761afac68 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.567341 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312ff66-996f-459f-be56-4a4761afac68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.580511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312ff66-996f-459f-be56-4a4761afac68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4312ff66-996f-459f-be56-4a4761afac68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:20.588837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:20.589770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:20.590577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 155/312] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:20.599412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312ff66-996f-459f-be56-4a4761afac68 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:20.599566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07951c8f-2d1c-42df-9bf2-841bc8623e67 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:20.634806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4ea1c-a5a5-4b2b-bcce-796cc6bffcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.635928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ea1c-a5a5-4b2b-bcce-796cc6bffcd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.636071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ea1c-a5a5-4b2b-bcce-796cc6bffcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.636294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ea1c-a5a5-4b2b-bcce-796cc6bffcd2 None admin] PATH_INFO: `/v3/system/users/aa009f1ee665404b9295a09fd76e19b8/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.637779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4ea1c-a5a5-4b2b-bcce-796cc6bffcd2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aa009f1ee665404b9295a09fd76e19b8, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:20.639181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4ea1c-a5a5-4b2b-bcce-796cc6bffcd2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:20.649903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 158/313] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Jun 23 08:07:20 2022] PUT /identity/v3/system/users/aa009f1ee665404b9295a09fd76e19b8/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:20.715025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312ff66-996f-459f-be56-4a4761afac68 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:20.754736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.755737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.756166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.756586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.757610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:20.758099 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:20.759704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:20.769965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 156/314] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:20.775795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4ea1c-a5a5-4b2b-bcce-796cc6bffcd2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:20.888168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4ea1c-a5a5-4b2b-bcce-796cc6bffcd2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:20.899948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:20.924044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ab27f-7ec7-4bee-8f9a-fef9659c4526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:20.924940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ab27f-7ec7-4bee-8f9a-fef9659c4526 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:20.925222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ab27f-7ec7-4bee-8f9a-fef9659c4526 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:20.925498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ab27f-7ec7-4bee-8f9a-fef9659c4526 None admin] PATH_INFO: `/v3/users/ed0adb9559924bfb8c7e9a6647a36cca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:20.926702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ab27f-7ec7-4bee-8f9a-fef9659c4526 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0adb9559924bfb8c7e9a6647a36cca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:20.928226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ab27f-7ec7-4bee-8f9a-fef9659c4526 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:20.930209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 159/315] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:20 2022] GET /identity/v3/users/ed0adb9559924bfb8c7e9a6647a36cca => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:20.935815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:20.992550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:21.043664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6ad5a-739f-4692-93d8-8934fa389144 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:21.078875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230d7ec-6627-4c05-a83f-9aabe2cad036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:21.079868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230d7ec-6627-4c05-a83f-9aabe2cad036 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:21.080354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230d7ec-6627-4c05-a83f-9aabe2cad036 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:21.080857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230d7ec-6627-4c05-a83f-9aabe2cad036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:21.152464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230d7ec-6627-4c05-a83f-9aabe2cad036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:21.154303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230d7ec-6627-4c05-a83f-9aabe2cad036 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:21.209534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:21.263969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 157/316] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:21.269113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ab27f-7ec7-4bee-8f9a-fef9659c4526 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:21.371201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ab27f-7ec7-4bee-8f9a-fef9659c4526 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:21.408287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795920b-8ac9-447e-9fde-ced65c01c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:21.409171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795920b-8ac9-447e-9fde-ced65c01c076 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:21.409458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795920b-8ac9-447e-9fde-ced65c01c076 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:21.409737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795920b-8ac9-447e-9fde-ced65c01c076 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:21.410934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795920b-8ac9-447e-9fde-ced65c01c076 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:21.412486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795920b-8ac9-447e-9fde-ced65c01c076 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:21.414651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 160/317] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:21 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:21.420393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230d7ec-6627-4c05-a83f-9aabe2cad036 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:21.525956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230d7ec-6627-4c05-a83f-9aabe2cad036 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:21.563139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba7ff3-c840-4cbb-96a9-b7ec7cdb8c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:21.563908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba7ff3-c840-4cbb-96a9-b7ec7cdb8c3c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:21.564193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba7ff3-c840-4cbb-96a9-b7ec7cdb8c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:21.564472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba7ff3-c840-4cbb-96a9-b7ec7cdb8c3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:21.565305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba7ff3-c840-4cbb-96a9-b7ec7cdb8c3c None admin] RBAC: Adding query filter params (role.id=e7d8e4fc6f35425ab4e5c87ed577de1f, scope.project.id=e44154970e4b4efda8f5cd260edbc63e, user.id=ed0adb9559924bfb8c7e9a6647a36cca) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:21.565620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba7ff3-c840-4cbb-96a9-b7ec7cdb8c3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:21.567076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba7ff3-c840-4cbb-96a9-b7ec7cdb8c3c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:21.573836 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 158/318] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:21 2022] GET /identity/v3/role_assignments?user.id=ed0adb9559924bfb8c7e9a6647a36cca&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=e7d8e4fc6f35425ab4e5c87ed577de1f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:22.170588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6ee41-cf9f-4935-bfae-98080318b242 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.171289 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6ee41-cf9f-4935-bfae-98080318b242 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.171771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6ee41-cf9f-4935-bfae-98080318b242 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.173741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 161/319] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:22.179029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfad47-8eb2-4110-8d1d-b080b979090c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.179296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfad47-8eb2-4110-8d1d-b080b979090c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.179710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfad47-8eb2-4110-8d1d-b080b979090c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.197492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8ccfad47-8eb2-4110-8d1d-b080b979090c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:22.198658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8ccfad47-8eb2-4110-8d1d-b080b979090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:22.210579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:22.270606 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 159/320] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:22.293203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6ee41-cf9f-4935-bfae-98080318b242 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:22.331966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd9e6-cb60-41ee-88b3-8ebbd291198b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.332279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd9e6-cb60-41ee-88b3-8ebbd291198b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.332532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd9e6-cb60-41ee-88b3-8ebbd291198b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.334116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 160/321] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:22.339094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be10fbe-be72-47e9-82c6-b175af73d312 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.339410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be10fbe-be72-47e9-82c6-b175af73d312 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.339693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be10fbe-be72-47e9-82c6-b175af73d312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.348163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:22.356963 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4be10fbe-be72-47e9-82c6-b175af73d312 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:22.358149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4be10fbe-be72-47e9-82c6-b175af73d312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:22.409345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6ee41-cf9f-4935-bfae-98080318b242 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:22.433595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 161/322] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:22.446461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39721b4-1e7b-496f-ad01-9ecaa221efd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:22.447587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39721b4-1e7b-496f-ad01-9ecaa221efd8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.448011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39721b4-1e7b-496f-ad01-9ecaa221efd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.448444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39721b4-1e7b-496f-ad01-9ecaa221efd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.451009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39721b4-1e7b-496f-ad01-9ecaa221efd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:22.452688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39721b4-1e7b-496f-ad01-9ecaa221efd8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:22.454565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 162/323] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:22.455802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be10fbe-be72-47e9-82c6-b175af73d312 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:22.461438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39721b4-1e7b-496f-ad01-9ecaa221efd8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:22.544372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:22.569267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be10fbe-be72-47e9-82c6-b175af73d312 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:22.571409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39721b4-1e7b-496f-ad01-9ecaa221efd8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:22.584162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:22.606743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53630ec6-aac0-4a25-86e8-112ba70292be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:22.608015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53630ec6-aac0-4a25-86e8-112ba70292be None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.608192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bcc13-59fc-4acd-b81b-b7c31a05aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:22.608580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53630ec6-aac0-4a25-86e8-112ba70292be None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.608867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bcc13-59fc-4acd-b81b-b7c31a05aae6 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.609153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bcc13-59fc-4acd-b81b-b7c31a05aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.609254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53630ec6-aac0-4a25-86e8-112ba70292be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.609427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bcc13-59fc-4acd-b81b-b7c31a05aae6 None admin] PATH_INFO: `/v3/groups` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.610149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bcc13-59fc-4acd-b81b-b7c31a05aae6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:22.611591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bcc13-59fc-4acd-b81b-b7c31a05aae6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:22.622949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53630ec6-aac0-4a25-86e8-112ba70292be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:22.629811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 163/324] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Jun 23 08:07:22 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-53630ec6-aac0-4a25-86e8-112ba70292be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:22.631152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:22.632471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:22.633304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 162/325] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:22.643755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bcc13-59fc-4acd-b81b-b7c31a05aae6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:22.744674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f789ed-d067-4d39-b609-890ef5c10d90 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.744990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f789ed-d067-4d39-b609-890ef5c10d90 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.745150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f789ed-d067-4d39-b609-890ef5c10d90 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.746536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 163/326] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:22.752381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb7f1d-4150-4a8a-981b-fe6e93254840 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.752697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb7f1d-4150-4a8a-981b-fe6e93254840 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.752940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb7f1d-4150-4a8a-981b-fe6e93254840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.754177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bcc13-59fc-4acd-b81b-b7c31a05aae6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:22.770240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fafb7f1d-4150-4a8a-981b-fe6e93254840 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:22.771662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fafb7f1d-4150-4a8a-981b-fe6e93254840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:22.796859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:22.797545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.797829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.798346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.798855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:22.799166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:22.800609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:22.809758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 164/327] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:22.815773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:22.844988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 164/328] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:22.853319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:22.869813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb7f1d-4150-4a8a-981b-fe6e93254840 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:22.941380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c306e-5920-4de7-99aa-2e4be69c7a00 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:22.962089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:22.977269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846cfe-e967-4d48-bc94-e4db105316d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:22.978165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846cfe-e967-4d48-bc94-e4db105316d0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:22.978446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846cfe-e967-4d48-bc94-e4db105316d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:22.978732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846cfe-e967-4d48-bc94-e4db105316d0 None admin] PATH_INFO: `/v3/users/b599f6ac2b794ca6aa6fc746f7824d60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:22.980040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a846cfe-e967-4d48-bc94-e4db105316d0 None admin] RBAC: Authorizing `identity:get_user(user_id=b599f6ac2b794ca6aa6fc746f7824d60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:22.981547 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a846cfe-e967-4d48-bc94-e4db105316d0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:22.983157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 165/329] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:22 2022] GET /identity/v3/users/b599f6ac2b794ca6aa6fc746f7824d60 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:22.988838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846cfe-e967-4d48-bc94-e4db105316d0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:23.003572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb7f1d-4150-4a8a-981b-fe6e93254840 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:23.051170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7d68f-1625-4c3e-a74e-83222267b1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:23.052024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7d68f-1625-4c3e-a74e-83222267b1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:23.052397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7d68f-1625-4c3e-a74e-83222267b1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:23.052882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7d68f-1625-4c3e-a74e-83222267b1e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:23.064988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7d68f-1625-4c3e-a74e-83222267b1e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-22e7d68f-1625-4c3e-a74e-83222267b1e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:23.073848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:23.074753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:23.075604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 165/330] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:23.080721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7d68f-1625-4c3e-a74e-83222267b1e3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:23.133200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846cfe-e967-4d48-bc94-e4db105316d0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:23.151859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:23.176191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec93944-f959-42c3-b2f8-8550766a9d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:23.177083 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec93944-f959-42c3-b2f8-8550766a9d2c None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:23.177393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec93944-f959-42c3-b2f8-8550766a9d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:23.177683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec93944-f959-42c3-b2f8-8550766a9d2c None admin] PATH_INFO: `/v3/system/users/b599f6ac2b794ca6aa6fc746f7824d60/roles/e7d8e4fc6f35425ab4e5c87ed577de1f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:23.179636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec93944-f959-42c3-b2f8-8550766a9d2c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b599f6ac2b794ca6aa6fc746f7824d60, role_id=e7d8e4fc6f35425ab4e5c87ed577de1f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:23.181683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec93944-f959-42c3-b2f8-8550766a9d2c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:23.195013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 166/331] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Jun 23 08:07:22 2022] PUT /identity/v3/system/users/b599f6ac2b794ca6aa6fc746f7824d60/roles/e7d8e4fc6f35425ab4e5c87ed577de1f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:23.200147 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7d68f-1625-4c3e-a74e-83222267b1e3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:23.237809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:23.248986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:23.249687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:23.249971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:23.250244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:23.251025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:23.251401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:23.252857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:23.264312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 166/332] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:23.270070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec93944-f959-42c3-b2f8-8550766a9d2c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:23.396894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec93944-f959-42c3-b2f8-8550766a9d2c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:23.440081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c698d-9481-4a37-92c4-2308e9e03cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:23.440970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c698d-9481-4a37-92c4-2308e9e03cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:23.441253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c698d-9481-4a37-92c4-2308e9e03cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:23.441528 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c698d-9481-4a37-92c4-2308e9e03cf7 None admin] PATH_INFO: `/v3/users/aa009f1ee665404b9295a09fd76e19b8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:23.442728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c698d-9481-4a37-92c4-2308e9e03cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=aa009f1ee665404b9295a09fd76e19b8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:23.444668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c698d-9481-4a37-92c4-2308e9e03cf7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:23.446641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 167/333] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:23 2022] GET /identity/v3/users/aa009f1ee665404b9295a09fd76e19b8 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:23.452217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:23.559916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace5289-e880-45ee-a698-1408b32d0e31 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:23.594400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0393b-2d58-4690-9a9d-5ca1caa19b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:23.595066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0393b-2d58-4690-9a9d-5ca1caa19b30 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:23.595376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0393b-2d58-4690-9a9d-5ca1caa19b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:23.595647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0393b-2d58-4690-9a9d-5ca1caa19b30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:23.596485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0393b-2d58-4690-9a9d-5ca1caa19b30 None admin] RBAC: Adding query filter params (role.id=af1e89bc75e74c9f819959d898d086a5, user.id=aa009f1ee665404b9295a09fd76e19b8, scope.system=all) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:23.596797 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0393b-2d58-4690-9a9d-5ca1caa19b30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:23.598259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0393b-2d58-4690-9a9d-5ca1caa19b30 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:23.611157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 167/334] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 23 08:07:23 2022] GET /identity/v3/role_assignments?user.id=aa009f1ee665404b9295a09fd76e19b8&scope.system=all&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:24.960727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d69cc6-06c2-4cd1-ba74-bc844811eb8b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:24.961257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d69cc6-06c2-4cd1-ba74-bc844811eb8b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:24.961436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d69cc6-06c2-4cd1-ba74-bc844811eb8b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:24.962831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 168/335] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:24.968295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5500d5-f8d3-4da1-a406-eff61aff2e6e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:24.968732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5500d5-f8d3-4da1-a406-eff61aff2e6e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:24.968853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5500d5-f8d3-4da1-a406-eff61aff2e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:24.993332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-dd5500d5-f8d3-4da1-a406-eff61aff2e6e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:24.994532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dd5500d5-f8d3-4da1-a406-eff61aff2e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:25.054414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:25.088493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 168/336] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:25.124894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d69cc6-06c2-4cd1-ba74-bc844811eb8b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:25.173891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:25.258952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d69cc6-06c2-4cd1-ba74-bc844811eb8b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:25.299326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770118d-24cf-4472-81c6-8adabc4d87ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:25.300156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770118d-24cf-4472-81c6-8adabc4d87ee None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:25.300438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770118d-24cf-4472-81c6-8adabc4d87ee None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:25.300713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770118d-24cf-4472-81c6-8adabc4d87ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:25.314631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770118d-24cf-4472-81c6-8adabc4d87ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d770118d-24cf-4472-81c6-8adabc4d87ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:25.323368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:25.324261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:25.325059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 169/337] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:25.330955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5500d5-f8d3-4da1-a406-eff61aff2e6e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:25.454034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5500d5-f8d3-4da1-a406-eff61aff2e6e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:25.493368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:25.494022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:25.494303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:25.494583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:25.495376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:25.495683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:25.497386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:25.507239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b886637-b252-4240-850f-171e33594a8f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:25.507239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b886637-b252-4240-850f-171e33594a8f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:25.507239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b886637-b252-4240-850f-171e33594a8f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:25.507847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 170/338] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:25.514925 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 169/339] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:25.516197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32118769-8c34-4d24-a55d-92fd2ae4fe51 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:25.516485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32118769-8c34-4d24-a55d-92fd2ae4fe51 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:25.516763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32118769-8c34-4d24-a55d-92fd2ae4fe51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:25.525158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:25.542614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32118769-8c34-4d24-a55d-92fd2ae4fe51 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:25.542614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32118769-8c34-4d24-a55d-92fd2ae4fe51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:25.599878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:25.627622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 171/340] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:25.678206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32118769-8c34-4d24-a55d-92fd2ae4fe51 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:25.709200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:25.725157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ed2d5-c202-4737-8b38-366b31472c0c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:25.797718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76362dbe-9ee0-4d11-bd75-8463871425c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:25.798819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76362dbe-9ee0-4d11-bd75-8463871425c0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:25.799338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76362dbe-9ee0-4d11-bd75-8463871425c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:25.799881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76362dbe-9ee0-4d11-bd75-8463871425c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:25.809403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32118769-8c34-4d24-a55d-92fd2ae4fe51 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:25.854180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448885f2-4000-410f-9c95-692e40e9bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:25.855054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448885f2-4000-410f-9c95-692e40e9bac4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:25.855801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448885f2-4000-410f-9c95-692e40e9bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:25.855801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448885f2-4000-410f-9c95-692e40e9bac4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:25.871082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448885f2-4000-410f-9c95-692e40e9bac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-448885f2-4000-410f-9c95-692e40e9bac4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:25.883045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:25.884054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:25.884836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 172/341] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:25.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448885f2-4000-410f-9c95-692e40e9bac4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:25.902215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76362dbe-9ee0-4d11-bd75-8463871425c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:25.904028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76362dbe-9ee0-4d11-bd75-8463871425c0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:25.968571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:26.001389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:26.020013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448885f2-4000-410f-9c95-692e40e9bac4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:26.058937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f065f-54bb-436b-ac42-d0165a1cca04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:26.059716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f065f-54bb-436b-ac42-d0165a1cca04 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.060006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f065f-54bb-436b-ac42-d0165a1cca04 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.060407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f065f-54bb-436b-ac42-d0165a1cca04 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.061173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f065f-54bb-436b-ac42-d0165a1cca04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:26.061485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f065f-54bb-436b-ac42-d0165a1cca04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:26.063177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f065f-54bb-436b-ac42-d0165a1cca04 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:26.073172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 173/342] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:26.074917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab86ba-0a10-4f68-b2a0-a590df395dc9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.075218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab86ba-0a10-4f68-b2a0-a590df395dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.075504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab86ba-0a10-4f68-b2a0-a590df395dc9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.077042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 174/343] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:26.085295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a19b-40e9-4ba8-882c-e747f07a1b32 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.085722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a19b-40e9-4ba8-882c-e747f07a1b32 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.086122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a19b-40e9-4ba8-882c-e747f07a1b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.103293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 170/344] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:26.104405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76362dbe-9ee0-4d11-bd75-8463871425c0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:26.110348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3586a19b-40e9-4ba8-882c-e747f07a1b32 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:26.111700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3586a19b-40e9-4ba8-882c-e747f07a1b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:26.196380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 175/345] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:26.199258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a19b-40e9-4ba8-882c-e747f07a1b32 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:26.290756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:26.316412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a19b-40e9-4ba8-882c-e747f07a1b32 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:26.344991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76362dbe-9ee0-4d11-bd75-8463871425c0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:26.384643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4863b5-4148-4be6-9a38-b4f14f3eb204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:26.385432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4863b5-4148-4be6-9a38-b4f14f3eb204 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.385721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4863b5-4148-4be6-9a38-b4f14f3eb204 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.385995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4863b5-4148-4be6-9a38-b4f14f3eb204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.397331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:26.428239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8cfd9-2323-423e-8dd6-fcaffbe29a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:26.429495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8cfd9-2323-423e-8dd6-fcaffbe29a5f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.429972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8cfd9-2323-423e-8dd6-fcaffbe29a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.430480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8cfd9-2323-423e-8dd6-fcaffbe29a5f None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.431939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8cfd9-2323-423e-8dd6-fcaffbe29a5f None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:26.433600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8cfd9-2323-423e-8dd6-fcaffbe29a5f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:26.435848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 176/346] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:26 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:26.436821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8cfd9-2323-423e-8dd6-fcaffbe29a5f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:26.466557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4863b5-4148-4be6-9a38-b4f14f3eb204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:26.468067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4863b5-4148-4be6-9a38-b4f14f3eb204 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:26.574943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 171/347] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:26.575810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4863b5-4148-4be6-9a38-b4f14f3eb204 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:26.580103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8cfd9-2323-423e-8dd6-fcaffbe29a5f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:26.618714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:26.628461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9721127-8dfe-45e4-aa32-93bc1236b993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:26.629296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9721127-8dfe-45e4-aa32-93bc1236b993 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.629578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9721127-8dfe-45e4-aa32-93bc1236b993 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.629851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9721127-8dfe-45e4-aa32-93bc1236b993 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.666647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9721127-8dfe-45e4-aa32-93bc1236b993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a9721127-8dfe-45e4-aa32-93bc1236b993 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:26.674837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:26.675912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 08:07:26.676723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:26.679190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 177/348] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:26.681018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f1d86-263a-401a-8c4f-dd05cd3f9b62 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.681527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f1d86-263a-401a-8c4f-dd05cd3f9b62 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.681999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f1d86-263a-401a-8c4f-dd05cd3f9b62 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.686064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 178/349] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:26.686064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f1d86-263a-401a-8c4f-dd05cd3f9b62 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:26.689104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4863b5-4148-4be6-9a38-b4f14f3eb204 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:26.693806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:26.737253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3f48c-118b-4206-a90e-8f02acdba69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:26.738130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3f48c-118b-4206-a90e-8f02acdba69d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.738408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3f48c-118b-4206-a90e-8f02acdba69d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.738677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3f48c-118b-4206-a90e-8f02acdba69d None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.746004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3f48c-118b-4206-a90e-8f02acdba69d None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:26.748863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3f48c-118b-4206-a90e-8f02acdba69d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:26.751706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 172/350] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:26 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:26.753636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df4197-4156-45b8-bddd-b7dba649612d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.754356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df4197-4156-45b8-bddd-b7dba649612d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.755133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df4197-4156-45b8-bddd-b7dba649612d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.757081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 173/351] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:26.759267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df4197-4156-45b8-bddd-b7dba649612d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:26.821624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f1d86-263a-401a-8c4f-dd05cd3f9b62 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:26.867752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9607bb8-3473-41b2-9ca6-0562e5f21b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:26.869243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9607bb8-3473-41b2-9ca6-0562e5f21b75 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.869700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9607bb8-3473-41b2-9ca6-0562e5f21b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.870112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9607bb8-3473-41b2-9ca6-0562e5f21b75 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.871531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9607bb8-3473-41b2-9ca6-0562e5f21b75 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:26.873980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9607bb8-3473-41b2-9ca6-0562e5f21b75 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:26.876429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 179/352] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:26 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:26.878048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf48b-ff3d-46ca-9565-ab3c9d3516fc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.878681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf48b-ff3d-46ca-9565-ab3c9d3516fc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.879193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf48b-ff3d-46ca-9565-ab3c9d3516fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.889757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df4197-4156-45b8-bddd-b7dba649612d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:26.891394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:26.902116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fccaf48b-ff3d-46ca-9565-ab3c9d3516fc None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:26.902116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fccaf48b-ff3d-46ca-9565-ab3c9d3516fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:26.935394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6e958-2e88-4716-b8c1-3cdc18ea6b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:26.935394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e958-2e88-4716-b8c1-3cdc18ea6b84 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.935918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e958-2e88-4716-b8c1-3cdc18ea6b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.936197 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e958-2e88-4716-b8c1-3cdc18ea6b84 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.937112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6e958-2e88-4716-b8c1-3cdc18ea6b84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:26.937413 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6e958-2e88-4716-b8c1-3cdc18ea6b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:26.939118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6e958-2e88-4716-b8c1-3cdc18ea6b84 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:26.952939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 174/353] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:26.954761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955579d1-e0ae-4480-a35e-cf34e3b09a57 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:26.956252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955579d1-e0ae-4480-a35e-cf34e3b09a57 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:26.956704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955579d1-e0ae-4480-a35e-cf34e3b09a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:26.981454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-955579d1-e0ae-4480-a35e-cf34e3b09a57 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:26.982680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-955579d1-e0ae-4480-a35e-cf34e3b09a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:26.995327 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 180/354] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:26.996509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaf48b-ff3d-46ca-9565-ab3c9d3516fc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.001236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:27.064532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 175/355] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:27.064532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955579d1-e0ae-4480-a35e-cf34e3b09a57 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.120765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaf48b-ff3d-46ca-9565-ab3c9d3516fc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.170727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:27.171872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.174586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.174586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.174586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] RBAC: Adding query filter params (group.id=fca0623470de40d3b2f9174e1b006f72, role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=20862a0ad40a41af9af9f940f3d4c544) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:27.174586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:27.175837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:27.177608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:27.186025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 181/356] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:26 2022] GET /identity/v3/role_assignments?group.id=fca0623470de40d3b2f9174e1b006f72&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:27.187122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.193332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955579d1-e0ae-4480-a35e-cf34e3b09a57 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.244589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20598aa6-49d7-4416-a56e-de9997ad8198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:27.245474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20598aa6-49d7-4416-a56e-de9997ad8198 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.245754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20598aa6-49d7-4416-a56e-de9997ad8198 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.246028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20598aa6-49d7-4416-a56e-de9997ad8198 None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.247631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20598aa6-49d7-4416-a56e-de9997ad8198 None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:27.250712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20598aa6-49d7-4416-a56e-de9997ad8198 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:27.252501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 176/357] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:27.253286 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20598aa6-49d7-4416-a56e-de9997ad8198 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.291259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:27.342894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0731e-d28b-41b3-a78d-9ee495447e74 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.380648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20598aa6-49d7-4416-a56e-de9997ad8198 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.388418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e57ba-96be-4670-a398-e95befdd416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:27.389393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e57ba-96be-4670-a398-e95befdd416b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.389680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e57ba-96be-4670-a398-e95befdd416b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.390076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e57ba-96be-4670-a398-e95befdd416b None admin] PATH_INFO: `/v3/users/b599f6ac2b794ca6aa6fc746f7824d60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.391493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e57ba-96be-4670-a398-e95befdd416b None admin] RBAC: Authorizing `identity:get_user(user_id=b599f6ac2b794ca6aa6fc746f7824d60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:27.393110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e57ba-96be-4670-a398-e95befdd416b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:27.394932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 182/358] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/users/b599f6ac2b794ca6aa6fc746f7824d60 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:27.395753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e57ba-96be-4670-a398-e95befdd416b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.428481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe742-3b68-4875-8ae8-17d96ea8d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:27.428481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe742-3b68-4875-8ae8-17d96ea8d636 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.428481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe742-3b68-4875-8ae8-17d96ea8d636 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.428481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe742-3b68-4875-8ae8-17d96ea8d636 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.439487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbe742-3b68-4875-8ae8-17d96ea8d636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-86cbe742-3b68-4875-8ae8-17d96ea8d636 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:27.448149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:27.449067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:27.449850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 177/359] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:27.450509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe742-3b68-4875-8ae8-17d96ea8d636 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.478173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:27.555345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e57ba-96be-4670-a398-e95befdd416b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.580325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe742-3b68-4875-8ae8-17d96ea8d636 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.595941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebbc46-dd27-4eaa-bc6b-4c42d951de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:27.596821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebbc46-dd27-4eaa-bc6b-4c42d951de3d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.597134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebbc46-dd27-4eaa-bc6b-4c42d951de3d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.597578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebbc46-dd27-4eaa-bc6b-4c42d951de3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.611959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebbc46-dd27-4eaa-bc6b-4c42d951de3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:27.619110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:27.620009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.620438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.620858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.622255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] RBAC: Adding query filter params (group.id=fca0623470de40d3b2f9174e1b006f72, role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=20862a0ad40a41af9af9f940f3d4c544) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-51ebbc46-dd27-4eaa-bc6b-4c42d951de3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:27.622372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:27.623366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:27.624184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:27.625534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:27.627545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:27.627660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:27.629372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 183/360] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:27.630143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebbc46-dd27-4eaa-bc6b-4c42d951de3d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.635726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 178/361] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/role_assignments?group.id=fca0623470de40d3b2f9174e1b006f72&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=798b7bcb53684ab49ae899f8902f444a => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:27.636800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.734397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:27.752510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422f681-e6d2-4263-8b74-e48e7b30c487 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.792258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:27.793325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.793815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.794314 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.795349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:27.795900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:27.797551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:27.809144 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 179/362] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:27.810430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.810886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebbc46-dd27-4eaa-bc6b-4c42d951de3d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.850089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f420c-a5cd-4063-9e1a-ce106b078bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:27.850767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f420c-a5cd-4063-9e1a-ce106b078bcc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.851043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f420c-a5cd-4063-9e1a-ce106b078bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.851345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f420c-a5cd-4063-9e1a-ce106b078bcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.852208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f420c-a5cd-4063-9e1a-ce106b078bcc None admin] RBAC: Adding query filter params (role.id=e7d8e4fc6f35425ab4e5c87ed577de1f, user.id=b599f6ac2b794ca6aa6fc746f7824d60, scope.system=all) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:27.852522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f420c-a5cd-4063-9e1a-ce106b078bcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:27.854015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f420c-a5cd-4063-9e1a-ce106b078bcc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:27.869908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 184/363] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/role_assignments?user.id=b599f6ac2b794ca6aa6fc746f7824d60&scope.system=all&role.id=e7d8e4fc6f35425ab4e5c87ed577de1f => generated 510 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:27.871381 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57c88d-d59a-4ef8-a47f-e215659ce002 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:27.871676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57c88d-d59a-4ef8-a47f-e215659ce002 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:27.872270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57c88d-d59a-4ef8-a47f-e215659ce002 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:27.873970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 185/364] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:27.874726 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57c88d-d59a-4ef8-a47f-e215659ce002 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:27.935722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:27.961729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327eb7c-eca3-4f7f-907f-d4d5dc4f5db3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:27.987239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57c88d-d59a-4ef8-a47f-e215659ce002 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:28.030858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8ff77-a8f5-4f2d-a30c-e65b740cd633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:28.031553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8ff77-a8f5-4f2d-a30c-e65b740cd633 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.031836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8ff77-a8f5-4f2d-a30c-e65b740cd633 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.032102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8ff77-a8f5-4f2d-a30c-e65b740cd633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.036905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d22453-1af2-4695-abc7-eb8dfceb1433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:28.037561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d22453-1af2-4695-abc7-eb8dfceb1433 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.037838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d22453-1af2-4695-abc7-eb8dfceb1433 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.038113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d22453-1af2-4695-abc7-eb8dfceb1433 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.038869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d22453-1af2-4695-abc7-eb8dfceb1433 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:28.039179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d22453-1af2-4695-abc7-eb8dfceb1433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:28.040622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d22453-1af2-4695-abc7-eb8dfceb1433 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:28.062596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 180/365] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:28.067152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27dbfe-fdf7-4c3c-afaf-98337fbc8945 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.067504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27dbfe-fdf7-4c3c-afaf-98337fbc8945 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.067742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27dbfe-fdf7-4c3c-afaf-98337fbc8945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.087378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1b27dbfe-fdf7-4c3c-afaf-98337fbc8945 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:28.088583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1b27dbfe-fdf7-4c3c-afaf-98337fbc8945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:28.092378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:28.149836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8ff77-a8f5-4f2d-a30c-e65b740cd633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:28.151445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8ff77-a8f5-4f2d-a30c-e65b740cd633 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:28.179337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 181/366] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:28.180501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27dbfe-fdf7-4c3c-afaf-98337fbc8945 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:28.240941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:28.300171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27dbfe-fdf7-4c3c-afaf-98337fbc8945 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:28.308847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 186/367] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:28.309859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8ff77-a8f5-4f2d-a30c-e65b740cd633 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:28.348233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961ec55-c01b-40e5-9cc5-63f5eaa6d5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:28.349022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ec55-c01b-40e5-9cc5-63f5eaa6d5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.349485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ec55-c01b-40e5-9cc5-63f5eaa6d5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.349759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ec55-c01b-40e5-9cc5-63f5eaa6d5b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.362663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:28.427463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961ec55-c01b-40e5-9cc5-63f5eaa6d5b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:28.429422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961ec55-c01b-40e5-9cc5-63f5eaa6d5b3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:28.449267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8ff77-a8f5-4f2d-a30c-e65b740cd633 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:28.495409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ece64-7f7c-46c5-a066-902e56ebe355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:28.496639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ece64-7f7c-46c5-a066-902e56ebe355 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.497378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ece64-7f7c-46c5-a066-902e56ebe355 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.498167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ece64-7f7c-46c5-a066-902e56ebe355 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.516509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ece64-7f7c-46c5-a066-902e56ebe355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c87ece64-7f7c-46c5-a066-902e56ebe355 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:28.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:28.528034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:28.529044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:28.533847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 187/368] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:28.534975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ece64-7f7c-46c5-a066-902e56ebe355 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:28.554031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:28.558795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 182/369] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:28.559771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961ec55-c01b-40e5-9cc5-63f5eaa6d5b3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:28.609317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:28.669792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ece64-7f7c-46c5-a066-902e56ebe355 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:28.684603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961ec55-c01b-40e5-9cc5-63f5eaa6d5b3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:28.716958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8829ba-36c4-45d1-94eb-92b77d02ea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:28.717849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8829ba-36c4-45d1-94eb-92b77d02ea9d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.718130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8829ba-36c4-45d1-94eb-92b77d02ea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.718407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8829ba-36c4-45d1-94eb-92b77d02ea9d None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.719644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8829ba-36c4-45d1-94eb-92b77d02ea9d None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:28.721112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8829ba-36c4-45d1-94eb-92b77d02ea9d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:28.723124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 188/370] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:28 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:28.723919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8829ba-36c4-45d1-94eb-92b77d02ea9d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:28.735231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:28.736359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.736646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.736923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.737687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:28.737999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:28.740081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:28.752708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 183/371] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:28.753902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:28.822442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:28.843824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8829ba-36c4-45d1-94eb-92b77d02ea9d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:28.890240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39160-1644-4864-9b35-f6d273b053c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:28.891125 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39160-1644-4864-9b35-f6d273b053c0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.891252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec5e65-d009-4334-9f4d-1702e1900039 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:28.891446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39160-1644-4864-9b35-f6d273b053c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.891708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39160-1644-4864-9b35-f6d273b053c0 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.892783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:28.892947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da39160-1644-4864-9b35-f6d273b053c0 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:28.894697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da39160-1644-4864-9b35-f6d273b053c0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:28.896967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 189/372] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:28 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:28.897748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39160-1644-4864-9b35-f6d273b053c0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:28.934648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87c89a-dc98-41f6-9986-7d0ac68d91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:28.935545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c89a-dc98-41f6-9986-7d0ac68d91d2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:28.935827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c89a-dc98-41f6-9986-7d0ac68d91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:28.936197 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c89a-dc98-41f6-9986-7d0ac68d91d2 None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:28.937468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87c89a-dc98-41f6-9986-7d0ac68d91d2 None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:28.938926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87c89a-dc98-41f6-9986-7d0ac68d91d2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:28.940873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 184/373] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:28 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:28.941653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87c89a-dc98-41f6-9986-7d0ac68d91d2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:29.034317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39160-1644-4864-9b35-f6d273b053c0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:29.054935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87c89a-dc98-41f6-9986-7d0ac68d91d2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:29.079728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb2d39-1739-4c8d-bf67-ba3b865f47b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:29.080910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb2d39-1739-4c8d-bf67-ba3b865f47b2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.081200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb2d39-1739-4c8d-bf67-ba3b865f47b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.081751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb2d39-1739-4c8d-bf67-ba3b865f47b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.101597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86c8f9-e241-40c9-b893-75860eb11bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:29.103199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86c8f9-e241-40c9-b893-75860eb11bed None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.103515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86c8f9-e241-40c9-b893-75860eb11bed None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.103796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86c8f9-e241-40c9-b893-75860eb11bed None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.104645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:29.107106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86c8f9-e241-40c9-b893-75860eb11bed None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:29.108717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86c8f9-e241-40c9-b893-75860eb11bed None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:29.112253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 185/374] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:28 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:29.113046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86c8f9-e241-40c9-b893-75860eb11bed None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:29.169217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:29.170476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb2d39-1739-4c8d-bf67-ba3b865f47b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:29.173302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb2d39-1739-4c8d-bf67-ba3b865f47b2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:29.232482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86c8f9-e241-40c9-b893-75860eb11bed None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:29.266466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:29.267126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.267517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.267710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.268551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] RBAC: Adding query filter params (group.id=fca0623470de40d3b2f9174e1b006f72, role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=e44154970e4b4efda8f5cd260edbc63e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:29.269128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:29.270313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:29.277518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 186/375] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:29 2022] GET /identity/v3/role_assignments?group.id=fca0623470de40d3b2f9174e1b006f72&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:29.278601 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:29.296445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 190/376] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:29.302274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb2d39-1739-4c8d-bf67-ba3b865f47b2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:29.385076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:29.386985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2af0a-81ef-44f1-b1a1-dab0f0b4ebb5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:29.407639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:29.427438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:29.428115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.428581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.428874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.429730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] RBAC: Adding query filter params (group.id=fca0623470de40d3b2f9174e1b006f72, role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=e44154970e4b4efda8f5cd260edbc63e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:29.430048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:29.431717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:29.438580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 187/377] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:29 2022] GET /identity/v3/role_assignments?group.id=fca0623470de40d3b2f9174e1b006f72&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=798b7bcb53684ab49ae899f8902f444a => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:29.501857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb2d39-1739-4c8d-bf67-ba3b865f47b2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:29.558557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392733-39e0-4a9e-bc8a-c4806ce2cc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:29.559406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392733-39e0-4a9e-bc8a-c4806ce2cc8c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.559688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392733-39e0-4a9e-bc8a-c4806ce2cc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.559961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392733-39e0-4a9e-bc8a-c4806ce2cc8c None admin] PATH_INFO: `/v3/projects/44e27258dd6f419b9b00f5d07551d7c9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.561202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce392733-39e0-4a9e-bc8a-c4806ce2cc8c None admin] RBAC: Authorizing `identity:get_project(project_id=44e27258dd6f419b9b00f5d07551d7c9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:29.562692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce392733-39e0-4a9e-bc8a-c4806ce2cc8c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:29.564707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 191/378] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:29 2022] GET /identity/v3/projects/44e27258dd6f419b9b00f5d07551d7c9 => generated 344 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:29.570189 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:29.710577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee731fb9-f226-4e28-b971-43e95c484abd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:29.757280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a48d88-c089-46d0-88b9-2615aeb911e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:29.758349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a48d88-c089-46d0-88b9-2615aeb911e8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.758630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a48d88-c089-46d0-88b9-2615aeb911e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.758901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a48d88-c089-46d0-88b9-2615aeb911e8 None admin] PATH_INFO: `/v3/groups/00fcad38810e4e77840c04b0ac44441c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.765347 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a48d88-c089-46d0-88b9-2615aeb911e8 None admin] RBAC: Authorizing `identity:get_group(group_id=00fcad38810e4e77840c04b0ac44441c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:29.766800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a48d88-c089-46d0-88b9-2615aeb911e8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:29.768716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 188/379] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:29 2022] GET /identity/v3/groups/00fcad38810e4e77840c04b0ac44441c => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:29.775942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392733-39e0-4a9e-bc8a-c4806ce2cc8c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:29.810289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd45b-79e8-4249-889b-e0a23020e858 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.810476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd45b-79e8-4249-889b-e0a23020e858 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.810750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd45b-79e8-4249-889b-e0a23020e858 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.812155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 189/380] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:29.817535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda12abe-f248-43b4-b681-c5eaabe80361 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.817957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda12abe-f248-43b4-b681-c5eaabe80361 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.818318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda12abe-f248-43b4-b681-c5eaabe80361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.845612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cda12abe-f248-43b4-b681-c5eaabe80361 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:29.846821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cda12abe-f248-43b4-b681-c5eaabe80361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:29.900752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:29.952128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392733-39e0-4a9e-bc8a-c4806ce2cc8c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:29.972398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:29.977582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 190/381] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:29.991196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e385249-2d44-4d6f-9805-7f39505db6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:29.991916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385249-2d44-4d6f-9805-7f39505db6eb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:29.992194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385249-2d44-4d6f-9805-7f39505db6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:29.992462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385249-2d44-4d6f-9805-7f39505db6eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:29.993296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e385249-2d44-4d6f-9805-7f39505db6eb None admin] RBAC: Adding query filter params (group.id=00fcad38810e4e77840c04b0ac44441c, role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=44e27258dd6f419b9b00f5d07551d7c9) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:29.993608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e385249-2d44-4d6f-9805-7f39505db6eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:29.995049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e385249-2d44-4d6f-9805-7f39505db6eb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:30.003129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 192/382] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:29 2022] GET /identity/v3/role_assignments?group.id=00fcad38810e4e77840c04b0ac44441c&scope.project.id=44e27258dd6f419b9b00f5d07551d7c9&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:30.019914 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda12abe-f248-43b4-b681-c5eaabe80361 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:30.125241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6535dd-380d-43e9-8062-edab1e59545e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:30.125595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6535dd-380d-43e9-8062-edab1e59545e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:30.125792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6535dd-380d-43e9-8062-edab1e59545e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:30.131617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 193/383] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:30.133777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626933eb-a1d7-483b-969e-b6a9f6d56b53 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:30.134060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626933eb-a1d7-483b-969e-b6a9f6d56b53 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:30.134334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626933eb-a1d7-483b-969e-b6a9f6d56b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:30.147051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda12abe-f248-43b4-b681-c5eaabe80361 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:30.162668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-626933eb-a1d7-483b-969e-b6a9f6d56b53 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:30.163615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-626933eb-a1d7-483b-969e-b6a9f6d56b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:30.186479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ef3e0-f636-4797-9ea1-a29ffc3728e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:30.187336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ef3e0-f636-4797-9ea1-a29ffc3728e9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:30.187622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ef3e0-f636-4797-9ea1-a29ffc3728e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:30.187894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ef3e0-f636-4797-9ea1-a29ffc3728e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:30.201101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ef3e0-f636-4797-9ea1-a29ffc3728e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c93ef3e0-f636-4797-9ea1-a29ffc3728e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:30.210373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:30.212718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:30.213796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 191/384] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:30.218193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ef3e0-f636-4797-9ea1-a29ffc3728e9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:30.243849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 194/385] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:30.247363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:30.269361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626933eb-a1d7-483b-969e-b6a9f6d56b53 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:30.350725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ef3e0-f636-4797-9ea1-a29ffc3728e9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:30.399085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:30.399722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:30.400004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:30.400287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:30.401049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:30.401358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:30.402749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:30.405625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626933eb-a1d7-483b-969e-b6a9f6d56b53 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:30.413321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 192/386] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:30.418466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:30.442273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:30.446943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38172a46-b655-4dbc-9c65-16d74b73e842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:30.447817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38172a46-b655-4dbc-9c65-16d74b73e842 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:30.448148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38172a46-b655-4dbc-9c65-16d74b73e842 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:30.448426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38172a46-b655-4dbc-9c65-16d74b73e842 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:30.461427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38172a46-b655-4dbc-9c65-16d74b73e842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-38172a46-b655-4dbc-9c65-16d74b73e842 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:30.470749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:30.471735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:30.472520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 195/387] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:30.479662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38172a46-b655-4dbc-9c65-16d74b73e842 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:30.605322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:30.643677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea93154-62a0-426c-a1b3-c4b8c81d923f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:30.692534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38172a46-b655-4dbc-9c65-16d74b73e842 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:30.719612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaf7ab-72b2-45f5-b521-adec51f4a6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:30.720494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaf7ab-72b2-45f5-b521-adec51f4a6fb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:30.720775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaf7ab-72b2-45f5-b521-adec51f4a6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:30.721044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaf7ab-72b2-45f5-b521-adec51f4a6fb None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:30.723323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaf7ab-72b2-45f5-b521-adec51f4a6fb None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:30.725005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaf7ab-72b2-45f5-b521-adec51f4a6fb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:30.726643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 193/388] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:30 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:30.732418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaf7ab-72b2-45f5-b521-adec51f4a6fb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:30.765071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:30.767189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:30.767524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:30.767796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:30.768558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:30.771159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:30.774037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:30.794454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 196/389] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:30.796381 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:30.863592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:30.936021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaf7ab-72b2-45f5-b521-adec51f4a6fb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:30.981104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdc4c4-3ecc-4ce5-949c-a3316d328ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:30.982048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc4c4-3ecc-4ce5-949c-a3316d328ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:30.982331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc4c4-3ecc-4ce5-949c-a3316d328ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:30.982603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc4c4-3ecc-4ce5-949c-a3316d328ce0 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:30.983833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdc4c4-3ecc-4ce5-949c-a3316d328ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:30.985278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdc4c4-3ecc-4ce5-949c-a3316d328ce0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:30.987399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 194/390] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:30 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:30.994898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdc4c4-3ecc-4ce5-949c-a3316d328ce0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:31.017742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:31.033202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87faf1e-a7e1-4807-a1ee-7f11a07ef219 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:31.113135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdc4c4-3ecc-4ce5-949c-a3316d328ce0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:31.119958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462752c9-6cdc-4a7e-9de9-b753092dfa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:31.121127 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462752c9-6cdc-4a7e-9de9-b753092dfa2a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:31.121609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462752c9-6cdc-4a7e-9de9-b753092dfa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:31.122378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462752c9-6cdc-4a7e-9de9-b753092dfa2a None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:31.123793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462752c9-6cdc-4a7e-9de9-b753092dfa2a None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:31.125462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462752c9-6cdc-4a7e-9de9-b753092dfa2a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:31.131357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 197/391] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:30 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:31.137435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462752c9-6cdc-4a7e-9de9-b753092dfa2a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:31.150145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa9d43-97f4-4448-aee4-b20bb3a099bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:31.151045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa9d43-97f4-4448-aee4-b20bb3a099bf None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:31.151353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa9d43-97f4-4448-aee4-b20bb3a099bf None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:31.151625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa9d43-97f4-4448-aee4-b20bb3a099bf None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544/groups/fca0623470de40d3b2f9174e1b006f72/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:31.159226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa9d43-97f4-4448-aee4-b20bb3a099bf None admin] RBAC: Authorizing `identity:create_grant(project_id=20862a0ad40a41af9af9f940f3d4c544, group_id=fca0623470de40d3b2f9174e1b006f72, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:31.159226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa9d43-97f4-4448-aee4-b20bb3a099bf None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:31.173118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 195/392] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 08:07:30 2022] PUT /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544/groups/fca0623470de40d3b2f9174e1b006f72/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:07:31.264157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:31.267439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462752c9-6cdc-4a7e-9de9-b753092dfa2a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:31.305355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1038fb-7606-4b8f-984e-0ca183811e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:31.306351 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1038fb-7606-4b8f-984e-0ca183811e40 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:31.306633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1038fb-7606-4b8f-984e-0ca183811e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:31.306904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1038fb-7606-4b8f-984e-0ca183811e40 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:31.308253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1038fb-7606-4b8f-984e-0ca183811e40 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:31.309721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1038fb-7606-4b8f-984e-0ca183811e40 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:31.311841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 198/393] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:31 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:31.317556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa9d43-97f4-4448-aee4-b20bb3a099bf None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:31.412709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:31.437423 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa9d43-97f4-4448-aee4-b20bb3a099bf None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:31.482168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ba57c-58c0-4d32-9b63-a22d6e3e3373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:31.483129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba57c-58c0-4d32-9b63-a22d6e3e3373 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:31.483434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba57c-58c0-4d32-9b63-a22d6e3e3373 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:31.483704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba57c-58c0-4d32-9b63-a22d6e3e3373 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544/groups/fca0623470de40d3b2f9174e1b006f72/roles/798b7bcb53684ab49ae899f8902f444a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:31.488718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ba57c-58c0-4d32-9b63-a22d6e3e3373 None admin] RBAC: Authorizing `identity:create_grant(project_id=20862a0ad40a41af9af9f940f3d4c544, group_id=fca0623470de40d3b2f9174e1b006f72, role_id=798b7bcb53684ab49ae899f8902f444a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:31.494355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ba57c-58c0-4d32-9b63-a22d6e3e3373 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:31.513056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 196/394] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 08:07:31 2022] PUT /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544/groups/fca0623470de40d3b2f9174e1b006f72/roles/798b7bcb53684ab49ae899f8902f444a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:31.997170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deae59-5eb8-4e08-b81e-7df233e9eec7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:31.997859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deae59-5eb8-4e08-b81e-7df233e9eec7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:31.998333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deae59-5eb8-4e08-b81e-7df233e9eec7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.000190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 199/395] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:32.006953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46b89-e1c4-429d-90d1-c6453e6d7a19 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.007258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46b89-e1c4-429d-90d1-c6453e6d7a19 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.007541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46b89-e1c4-429d-90d1-c6453e6d7a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.029797 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-43f46b89-e1c4-429d-90d1-c6453e6d7a19 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:32.031001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-43f46b89-e1c4-429d-90d1-c6453e6d7a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:32.112909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 197/396] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:32.152737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deae59-5eb8-4e08-b81e-7df233e9eec7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:32.177488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5e25b-e4b4-4abd-b778-d288d1da5a8c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.178057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5e25b-e4b4-4abd-b778-d288d1da5a8c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.178057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5e25b-e4b4-4abd-b778-d288d1da5a8c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.180198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 198/397] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:32.185577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84fefc-556c-4e55-b870-9ee7adef666c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.185913 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84fefc-556c-4e55-b870-9ee7adef666c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.186202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84fefc-556c-4e55-b870-9ee7adef666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.205790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8c84fefc-556c-4e55-b870-9ee7adef666c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:32.206986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8c84fefc-556c-4e55-b870-9ee7adef666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:32.274706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:32.281767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 199/398] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:32.298287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deae59-5eb8-4e08-b81e-7df233e9eec7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:32.308457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84fefc-556c-4e55-b870-9ee7adef666c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:32.356907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17368c04-3a99-43c4-8174-1b4197a06866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:32.357815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368c04-3a99-43c4-8174-1b4197a06866 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.358093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368c04-3a99-43c4-8174-1b4197a06866 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.358364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368c04-3a99-43c4-8174-1b4197a06866 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.377893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17368c04-3a99-43c4-8174-1b4197a06866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-17368c04-3a99-43c4-8174-1b4197a06866 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:32.390052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:32.392334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:32.394241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 200/399] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:32.404247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17368c04-3a99-43c4-8174-1b4197a06866 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:32.416202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84fefc-556c-4e55-b870-9ee7adef666c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:32.445020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:32.452195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a828ff-fd06-42c6-a7e0-140569a6f6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:32.453030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a828ff-fd06-42c6-a7e0-140569a6f6ca None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.453310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a828ff-fd06-42c6-a7e0-140569a6f6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.453645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a828ff-fd06-42c6-a7e0-140569a6f6ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.464951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a828ff-fd06-42c6-a7e0-140569a6f6ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-68a828ff-fd06-42c6-a7e0-140569a6f6ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:32.473195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:32.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:32.477480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 200/400] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:32.490634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a828ff-fd06-42c6-a7e0-140569a6f6ca None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:32.545880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:32.563549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17368c04-3a99-43c4-8174-1b4197a06866 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:32.591793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a828ff-fd06-42c6-a7e0-140569a6f6ca None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:32.622903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:32.623630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.623912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.624187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.624951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:32.625260 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:32.626675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:32.629097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:32.629821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.630180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.630464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.631685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:32.632002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:32.633494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:32.641878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 201/401] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:32.642617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 201/402] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:32.648088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:32.648604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:32.761163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45610208-8852-40c8-b946-75e6499d78f3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:32.770190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:32.828681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ea2c3-515c-4f8e-804f-e24e1519fb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:32.829498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea2c3-515c-4f8e-804f-e24e1519fb90 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.829782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea2c3-515c-4f8e-804f-e24e1519fb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.830064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea2c3-515c-4f8e-804f-e24e1519fb90 None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.831280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ea2c3-515c-4f8e-804f-e24e1519fb90 None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:32.832746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ea2c3-515c-4f8e-804f-e24e1519fb90 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:32.834638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 202/403] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:32 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:32.841138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779482b1-a81c-4371-8b50-10719f1db20a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.841422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779482b1-a81c-4371-8b50-10719f1db20a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.841701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779482b1-a81c-4371-8b50-10719f1db20a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.843144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 203/404] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:32.847820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059be4-22f5-4fd9-8193-780ed80e3b5e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.848108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059be4-22f5-4fd9-8193-780ed80e3b5e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.848387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059be4-22f5-4fd9-8193-780ed80e3b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.860722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211a3d-fde9-4066-94fa-5e90239c730e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:32.865074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a5059be4-22f5-4fd9-8193-780ed80e3b5e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:32.866275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a5059be4-22f5-4fd9-8193-780ed80e3b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:32.882785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:32.897973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ffc5a-314a-47f1-9156-65621714c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:32.898859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ffc5a-314a-47f1-9156-65621714c3af None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:32.899178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ffc5a-314a-47f1-9156-65621714c3af None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:32.899500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ffc5a-314a-47f1-9156-65621714c3af None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:32.901084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ffc5a-314a-47f1-9156-65621714c3af None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:32.902616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ffc5a-314a-47f1-9156-65621714c3af None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:32.904406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 202/405] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:32 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:32.905223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ffc5a-314a-47f1-9156-65621714c3af None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:32.939350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 204/406] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:32.940239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5059be4-22f5-4fd9-8193-780ed80e3b5e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:33.007339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ffc5a-314a-47f1-9156-65621714c3af None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:33.041393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcb448-e1cd-4d56-9c20-7a77aeb91f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:33.042313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcb448-e1cd-4d56-9c20-7a77aeb91f80 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.042596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcb448-e1cd-4d56-9c20-7a77aeb91f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.042866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcb448-e1cd-4d56-9c20-7a77aeb91f80 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.043651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:33.044085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcb448-e1cd-4d56-9c20-7a77aeb91f80 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:33.045521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcb448-e1cd-4d56-9c20-7a77aeb91f80 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:33.051983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 203/407] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:32 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:33.053058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcb448-e1cd-4d56-9c20-7a77aeb91f80 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:33.055831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5059be4-22f5-4fd9-8193-780ed80e3b5e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:33.092153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f786cb-c90c-4921-a999-24c1835436a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:33.092947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f786cb-c90c-4921-a999-24c1835436a0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.093227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f786cb-c90c-4921-a999-24c1835436a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.093503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f786cb-c90c-4921-a999-24c1835436a0 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.094783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f786cb-c90c-4921-a999-24c1835436a0 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:33.096270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f786cb-c90c-4921-a999-24c1835436a0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:33.098152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 205/408] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:32 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:33.098903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f786cb-c90c-4921-a999-24c1835436a0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:33.112039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:33.161127 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcb448-e1cd-4d56-9c20-7a77aeb91f80 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:33.197796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711d4ad-5207-465d-b3ec-7eee9e831b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:33.198688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711d4ad-5207-465d-b3ec-7eee9e831b98 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.199040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711d4ad-5207-465d-b3ec-7eee9e831b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.199411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711d4ad-5207-465d-b3ec-7eee9e831b98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.210504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711d4ad-5207-465d-b3ec-7eee9e831b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:33.217725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f786cb-c90c-4921-a999-24c1835436a0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4711d4ad-5207-465d-b3ec-7eee9e831b98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:33.218552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:33.219451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:33.220709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 204/409] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:33.222325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711d4ad-5207-465d-b3ec-7eee9e831b98 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:33.268434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b961e1-ac3d-454c-992f-22c56eef2d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:33.269328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b961e1-ac3d-454c-992f-22c56eef2d68 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.269623 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b961e1-ac3d-454c-992f-22c56eef2d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.269923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b961e1-ac3d-454c-992f-22c56eef2d68 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e/groups/fca0623470de40d3b2f9174e1b006f72/roles/798b7bcb53684ab49ae899f8902f444a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.272893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b961e1-ac3d-454c-992f-22c56eef2d68 None admin] RBAC: Authorizing `identity:create_grant(project_id=e44154970e4b4efda8f5cd260edbc63e, group_id=fca0623470de40d3b2f9174e1b006f72, role_id=798b7bcb53684ab49ae899f8902f444a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:33.274866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b961e1-ac3d-454c-992f-22c56eef2d68 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:33.294131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 206/410] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 08:07:33 2022] PUT /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e/groups/fca0623470de40d3b2f9174e1b006f72/roles/798b7bcb53684ab49ae899f8902f444a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:33.295434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b961e1-ac3d-454c-992f-22c56eef2d68 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:33.317249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:33.392858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711d4ad-5207-465d-b3ec-7eee9e831b98 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:33.430452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:33.441226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b961e1-ac3d-454c-992f-22c56eef2d68 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:33.451616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3b106-1dd3-47b1-b002-3b2c9f20ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:33.452521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3b106-1dd3-47b1-b002-3b2c9f20ef55 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.452798 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3b106-1dd3-47b1-b002-3b2c9f20ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.453071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3b106-1dd3-47b1-b002-3b2c9f20ef55 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e/groups/fca0623470de40d3b2f9174e1b006f72/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.459237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3b106-1dd3-47b1-b002-3b2c9f20ef55 None admin] RBAC: Authorizing `identity:create_grant(project_id=e44154970e4b4efda8f5cd260edbc63e, group_id=fca0623470de40d3b2f9174e1b006f72, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:33.459237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3b106-1dd3-47b1-b002-3b2c9f20ef55 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:33.473580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 205/411] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 08:07:33 2022] PUT /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e/groups/fca0623470de40d3b2f9174e1b006f72/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:33.547354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:33.548319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.548778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.549198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.550111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:33.550568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:33.552397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:33.567098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 207/412] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:33.572628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3b106-1dd3-47b1-b002-3b2c9f20ef55 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:33.699340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3b106-1dd3-47b1-b002-3b2c9f20ef55 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:33.734175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44752-ddf4-4be3-8a1e-619a64003ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:33.735169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44752-ddf4-4be3-8a1e-619a64003ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.735481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44752-ddf4-4be3-8a1e-619a64003ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.735756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44752-ddf4-4be3-8a1e-619a64003ab4 None admin] PATH_INFO: `/v3/groups/00fcad38810e4e77840c04b0ac44441c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.736936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d44752-ddf4-4be3-8a1e-619a64003ab4 None admin] RBAC: Authorizing `identity:get_group(group_id=00fcad38810e4e77840c04b0ac44441c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:33.738397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d44752-ddf4-4be3-8a1e-619a64003ab4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:33.740623 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 206/413] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:33 2022] GET /identity/v3/groups/00fcad38810e4e77840c04b0ac44441c => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:33.745659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:33.778581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e372c-5668-4151-be8f-a25e5de7639f None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.779249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e372c-5668-4151-be8f-a25e5de7639f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.779732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e372c-5668-4151-be8f-a25e5de7639f None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.782464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 207/414] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:33.788714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f18ad6-e08b-4c57-837e-bd105be2222f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.789283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f18ad6-e08b-4c57-837e-bd105be2222f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.789760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f18ad6-e08b-4c57-837e-bd105be2222f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.813555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-51f18ad6-e08b-4c57-837e-bd105be2222f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:33.814726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-51f18ad6-e08b-4c57-837e-bd105be2222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:33.840284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:33.866858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681b732-364d-478b-a7b4-1c486db85000 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:33.906344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5573241-c5bc-45b5-a954-dcece615db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:33.907235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5573241-c5bc-45b5-a954-dcece615db41 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:33.907521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5573241-c5bc-45b5-a954-dcece615db41 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:33.907797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5573241-c5bc-45b5-a954-dcece615db41 None admin] PATH_INFO: `/v3/projects/44e27258dd6f419b9b00f5d07551d7c9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:33.909256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5573241-c5bc-45b5-a954-dcece615db41 None admin] RBAC: Authorizing `identity:get_project(project_id=44e27258dd6f419b9b00f5d07551d7c9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:33.910729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5573241-c5bc-45b5-a954-dcece615db41 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:33.913175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 208/415] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:33 2022] GET /identity/v3/projects/44e27258dd6f419b9b00f5d07551d7c9 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:33.918718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5573241-c5bc-45b5-a954-dcece615db41 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:33.934464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:34.011823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 208/416] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:34.034955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f18ad6-e08b-4c57-837e-bd105be2222f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:34.035762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5573241-c5bc-45b5-a954-dcece615db41 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:34.074448 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb8877-c035-4fb0-809a-a2d34df2f38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:34.075548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8877-c035-4fb0-809a-a2d34df2f38f None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:34.075842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8877-c035-4fb0-809a-a2d34df2f38f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:34.076163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8877-c035-4fb0-809a-a2d34df2f38f None admin] PATH_INFO: `/v3/projects/44e27258dd6f419b9b00f5d07551d7c9/groups/00fcad38810e4e77840c04b0ac44441c/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:34.078241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb8877-c035-4fb0-809a-a2d34df2f38f None admin] RBAC: Authorizing `identity:create_grant(project_id=44e27258dd6f419b9b00f5d07551d7c9, group_id=00fcad38810e4e77840c04b0ac44441c, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:34.079788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb8877-c035-4fb0-809a-a2d34df2f38f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:34.096694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 209/417] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 08:07:33 2022] PUT /identity/v3/projects/44e27258dd6f419b9b00f5d07551d7c9/groups/00fcad38810e4e77840c04b0ac44441c/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:34.142265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6625081-c75d-4a90-b529-e0aac775aa1d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:34.142616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6625081-c75d-4a90-b529-e0aac775aa1d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:34.142924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6625081-c75d-4a90-b529-e0aac775aa1d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:34.144609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 210/418] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:34.150408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73b135-c92f-490a-beeb-d8afd4e047c4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:34.150966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73b135-c92f-490a-beeb-d8afd4e047c4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:34.151530 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73b135-c92f-490a-beeb-d8afd4e047c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:34.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:34.171384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0b73b135-c92f-490a-beeb-d8afd4e047c4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:34.173805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0b73b135-c92f-490a-beeb-d8afd4e047c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:34.221951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f18ad6-e08b-4c57-837e-bd105be2222f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:34.264759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 211/419] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:34.291130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73b135-c92f-490a-beeb-d8afd4e047c4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:34.308012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda6f7e-9db8-418c-b309-4f01970c7ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:34.309140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda6f7e-9db8-418c-b309-4f01970c7ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:34.309249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda6f7e-9db8-418c-b309-4f01970c7ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:34.309478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda6f7e-9db8-418c-b309-4f01970c7ac8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:34.323162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda6f7e-9db8-418c-b309-4f01970c7ac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4fda6f7e-9db8-418c-b309-4f01970c7ac8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:34.330966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:34.331884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:34.332685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 209/420] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:34.338657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda6f7e-9db8-418c-b309-4f01970c7ac8 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:34.375461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:34.418158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73b135-c92f-490a-beeb-d8afd4e047c4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:34.450516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda6f7e-9db8-418c-b309-4f01970c7ac8 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:34.460102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a224c0-4df6-4757-9b0d-91c474742336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:34.461538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a224c0-4df6-4757-9b0d-91c474742336 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:34.462340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a224c0-4df6-4757-9b0d-91c474742336 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:34.462949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a224c0-4df6-4757-9b0d-91c474742336 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:34.470622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:34.482856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a224c0-4df6-4757-9b0d-91c474742336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b2a224c0-4df6-4757-9b0d-91c474742336 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:34.501072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:34.502015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:34.502811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:34.503926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:34.504423 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:34.504702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:34.504976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:34.506149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:34.506461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:34.507043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 212/421] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:34.507973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:34.519304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a224c0-4df6-4757-9b0d-91c474742336 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:34.526814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 210/422] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:34.534259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:34.693473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a224c0-4df6-4757-9b0d-91c474742336 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:34.707288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:34.717031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a13a-5d9b-4929-af3b-836571aa8042 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:34.758134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:34.758817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:34.759096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:34.759407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:34.760158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:34.760465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:34.761942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:34.769584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ac5e5-6f55-4762-a8aa-590df032aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:34.770170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ac5e5-6f55-4762-a8aa-590df032aec1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:34.770462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ac5e5-6f55-4762-a8aa-590df032aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:34.770737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ac5e5-6f55-4762-a8aa-590df032aec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:34.772380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 213/423] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:34.780160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:34.893286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:34.926656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ac5e5-6f55-4762-a8aa-590df032aec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:34.929279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ac5e5-6f55-4762-a8aa-590df032aec1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:34.977718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e0af2-7d4c-4ae7-b98f-6aa659bf3d5f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:35.042226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a011769-20db-44fa-8c6e-fef02fe9d025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:35.043242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a011769-20db-44fa-8c6e-fef02fe9d025 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.043669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a011769-20db-44fa-8c6e-fef02fe9d025 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.044105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a011769-20db-44fa-8c6e-fef02fe9d025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.096379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:35.107944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 211/424] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:35.113079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ac5e5-6f55-4762-a8aa-590df032aec1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:35.131455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a011769-20db-44fa-8c6e-fef02fe9d025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:35.133123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a011769-20db-44fa-8c6e-fef02fe9d025 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:35.195575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:35.237413 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ac5e5-6f55-4762-a8aa-590df032aec1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:35.246206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 214/425] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:35.248088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a011769-20db-44fa-8c6e-fef02fe9d025 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:35.301324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee89d-fd40-402d-b025-63e7573e9168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:35.302003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee89d-fd40-402d-b025-63e7573e9168 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.302479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee89d-fd40-402d-b025-63e7573e9168 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.302958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee89d-fd40-402d-b025-63e7573e9168 None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.303869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ee89d-fd40-402d-b025-63e7573e9168 None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:35.305327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ee89d-fd40-402d-b025-63e7573e9168 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:35.313375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 212/426] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:35 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:35.318564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee89d-fd40-402d-b025-63e7573e9168 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:35.357264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a011769-20db-44fa-8c6e-fef02fe9d025 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:35.402031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4ea56-3218-4eba-b4e9-ee2d1e152e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:35.403288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ea56-3218-4eba-b4e9-ee2d1e152e7b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.403873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ea56-3218-4eba-b4e9-ee2d1e152e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.405012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ea56-3218-4eba-b4e9-ee2d1e152e7b None admin] PATH_INFO: `/v3/projects/20862a0ad40a41af9af9f940f3d4c544` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.407888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4ea56-3218-4eba-b4e9-ee2d1e152e7b None admin] RBAC: Authorizing `identity:get_project(project_id=20862a0ad40a41af9af9f940f3d4c544)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:35.409781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4ea56-3218-4eba-b4e9-ee2d1e152e7b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:35.419020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 215/427] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:35 2022] GET /identity/v3/projects/20862a0ad40a41af9af9f940f3d4c544 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:35.423037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4ea56-3218-4eba-b4e9-ee2d1e152e7b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:35.449106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:35.481136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:35.545828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee89d-fd40-402d-b025-63e7573e9168 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:35.616622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c7ff2-a31e-48a4-9cc9-ebfbb7bef892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:35.619728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c7ff2-a31e-48a4-9cc9-ebfbb7bef892 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.622560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c7ff2-a31e-48a4-9cc9-ebfbb7bef892 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.623053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c7ff2-a31e-48a4-9cc9-ebfbb7bef892 None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.624572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c7ff2-a31e-48a4-9cc9-ebfbb7bef892 None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:35.626252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c7ff2-a31e-48a4-9cc9-ebfbb7bef892 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:35.630233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 213/428] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:35 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:35.636713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c7ff2-a31e-48a4-9cc9-ebfbb7bef892 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:35.652871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4ea56-3218-4eba-b4e9-ee2d1e152e7b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:35.722389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aed956-9ce7-45ed-9ee6-ed467b242245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:35.723281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aed956-9ce7-45ed-9ee6-ed467b242245 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.723564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aed956-9ce7-45ed-9ee6-ed467b242245 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.723836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aed956-9ce7-45ed-9ee6-ed467b242245 None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.725793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aed956-9ce7-45ed-9ee6-ed467b242245 None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:35.727274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aed956-9ce7-45ed-9ee6-ed467b242245 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:35.728902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 216/429] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:35 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:35.734721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aed956-9ce7-45ed-9ee6-ed467b242245 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:35.829997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c7ff2-a31e-48a4-9cc9-ebfbb7bef892 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:35.872897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:35.886344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aed956-9ce7-45ed-9ee6-ed467b242245 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:35.913090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:35.926516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3177c0-ad4f-4903-b308-df270d8e8560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:35.927686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3177c0-ad4f-4903-b308-df270d8e8560 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.928168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3177c0-ad4f-4903-b308-df270d8e8560 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.928712 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3177c0-ad4f-4903-b308-df270d8e8560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.929804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3177c0-ad4f-4903-b308-df270d8e8560 None admin] RBAC: Adding query filter params (group.id=fca0623470de40d3b2f9174e1b006f72, role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=20862a0ad40a41af9af9f940f3d4c544) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:35.930362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3177c0-ad4f-4903-b308-df270d8e8560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:35.932331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3177c0-ad4f-4903-b308-df270d8e8560 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:35.936067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c55d3-a212-41c8-a122-e74ce43c9a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:35.936598 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c55d3-a212-41c8-a122-e74ce43c9a00 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.936887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c55d3-a212-41c8-a122-e74ce43c9a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.937169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c55d3-a212-41c8-a122-e74ce43c9a00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.938025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c55d3-a212-41c8-a122-e74ce43c9a00 None admin] RBAC: Adding query filter params (group.id=fca0623470de40d3b2f9174e1b006f72, role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=20862a0ad40a41af9af9f940f3d4c544) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:35.938333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c55d3-a212-41c8-a122-e74ce43c9a00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:35.939807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c55d3-a212-41c8-a122-e74ce43c9a00 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:35.940885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 217/430] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:35 2022] GET /identity/v3/role_assignments?group.id=fca0623470de40d3b2f9174e1b006f72&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=798b7bcb53684ab49ae899f8902f444a => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:35.948353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fe769-64e1-4022-95fb-647325bd3ad2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.948675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fe769-64e1-4022-95fb-647325bd3ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.948916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fe769-64e1-4022-95fb-647325bd3ad2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.950475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 218/431] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:35.958363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 214/432] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:35 2022] GET /identity/v3/role_assignments?group.id=fca0623470de40d3b2f9174e1b006f72&scope.project.id=20862a0ad40a41af9af9f940f3d4c544&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 611 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:35.960772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e89b5-d147-4ff0-ba35-a0736d4b6646 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.961066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e89b5-d147-4ff0-ba35-a0736d4b6646 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.961346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e89b5-d147-4ff0-ba35-a0736d4b6646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.973301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b2e1d-ac34-4718-a0c0-eafe1cf63ae5 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.973626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b2e1d-ac34-4718-a0c0-eafe1cf63ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.973858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b2e1d-ac34-4718-a0c0-eafe1cf63ae5 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.975781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 215/433] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:35.981615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e977c4e-2102-4bae-993c-40aaefb2ec19 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:35.981926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e977c4e-2102-4bae-993c-40aaefb2ec19 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:35.982174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e977c4e-2102-4bae-993c-40aaefb2ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:35.997213 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-250e89b5-d147-4ff0-ba35-a0736d4b6646 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:35.998448 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-250e89b5-d147-4ff0-ba35-a0736d4b6646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:36.014520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6e977c4e-2102-4bae-993c-40aaefb2ec19 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:36.015782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6e977c4e-2102-4bae-993c-40aaefb2ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:36.113281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 216/434] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:36.138837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e977c4e-2102-4bae-993c-40aaefb2ec19 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:36.143848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 219/435] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:36.169183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e89b5-d147-4ff0-ba35-a0736d4b6646 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:36.271506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e977c4e-2102-4bae-993c-40aaefb2ec19 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:36.281915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:36.301547 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e89b5-d147-4ff0-ba35-a0736d4b6646 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:36.318140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:36.328627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2baea-7f41-4222-86cf-8bce47ba056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:36.329450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2baea-7f41-4222-86cf-8bce47ba056d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:36.329733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2baea-7f41-4222-86cf-8bce47ba056d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:36.330006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2baea-7f41-4222-86cf-8bce47ba056d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:36.342536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e2baea-7f41-4222-86cf-8bce47ba056d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-40e2baea-7f41-4222-86cf-8bce47ba056d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:36.350859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:36.351781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:36.352604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:36.353527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16e0d-7f23-457e-a39e-9b670438e87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:36.353632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 217/436] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:36.358396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16e0d-7f23-457e-a39e-9b670438e87b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:36.358682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16e0d-7f23-457e-a39e-9b670438e87b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:36.358959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16e0d-7f23-457e-a39e-9b670438e87b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:36.359065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2baea-7f41-4222-86cf-8bce47ba056d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:36.375869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16e0d-7f23-457e-a39e-9b670438e87b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-40e16e0d-7f23-457e-a39e-9b670438e87b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:36.385439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:36.386570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 08:07:36.387639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:36.389760 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 220/437] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:36.397291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16e0d-7f23-457e-a39e-9b670438e87b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:36.503034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2baea-7f41-4222-86cf-8bce47ba056d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:36.538468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16e0d-7f23-457e-a39e-9b670438e87b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:36.555683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972a6e4-c22e-4163-a21f-f8175ba048b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:36.556495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972a6e4-c22e-4163-a21f-f8175ba048b0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:36.556773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972a6e4-c22e-4163-a21f-f8175ba048b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:36.557019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972a6e4-c22e-4163-a21f-f8175ba048b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:36.557790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972a6e4-c22e-4163-a21f-f8175ba048b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:36.558093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972a6e4-c22e-4163-a21f-f8175ba048b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:36.559618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972a6e4-c22e-4163-a21f-f8175ba048b0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:36.571258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 218/438] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:36.572519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b724-a3ca-4750-8798-2a622511d114 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:36.572811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b724-a3ca-4750-8798-2a622511d114 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:36.573144 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b724-a3ca-4750-8798-2a622511d114 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:36.574379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 219/439] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:36.577401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4b724-a3ca-4750-8798-2a622511d114 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:36.586896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b9aa9-07e9-42db-984e-52549d92d5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:36.587607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b9aa9-07e9-42db-984e-52549d92d5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:36.587884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b9aa9-07e9-42db-984e-52549d92d5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:36.588151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b9aa9-07e9-42db-984e-52549d92d5b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:36.588912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b9aa9-07e9-42db-984e-52549d92d5b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:36.589700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b9aa9-07e9-42db-984e-52549d92d5b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:36.591112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b9aa9-07e9-42db-984e-52549d92d5b7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:36.600506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 221/440] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:07:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:36.603495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d118f6-446d-4146-81dd-aa51a2066f49 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:36.603780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d118f6-446d-4146-81dd-aa51a2066f49 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:36.604060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d118f6-446d-4146-81dd-aa51a2066f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:36.625046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b8d118f6-446d-4146-81dd-aa51a2066f49 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:36.626350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b8d118f6-446d-4146-81dd-aa51a2066f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:36.630466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:36.635965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:36.693618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4b724-a3ca-4750-8798-2a622511d114 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:36.697617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 222/441] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:36.698290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d118f6-446d-4146-81dd-aa51a2066f49 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:36.730769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95940cd-52fc-448d-a710-6f05f45c06ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:36.731451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95940cd-52fc-448d-a710-6f05f45c06ea None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:36.731737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95940cd-52fc-448d-a710-6f05f45c06ea None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:36.732009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95940cd-52fc-448d-a710-6f05f45c06ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:36.795170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d118f6-446d-4146-81dd-aa51a2066f49 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:36.797590 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95940cd-52fc-448d-a710-6f05f45c06ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:36.799071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95940cd-52fc-448d-a710-6f05f45c06ea None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:36.828680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f0981-b67d-4286-b778-070202075c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:36.829325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f0981-b67d-4286-b778-070202075c65 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:36.829613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f0981-b67d-4286-b778-070202075c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:36.829893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f0981-b67d-4286-b778-070202075c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:36.840108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:36.843984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:36.895632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 220/442] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:36.898954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f0981-b67d-4286-b778-070202075c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:36.899235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95940cd-52fc-448d-a710-6f05f45c06ea None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:36.900540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f0981-b67d-4286-b778-070202075c65 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:36.997523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95940cd-52fc-448d-a710-6f05f45c06ea None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:36.997954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 223/443] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:37.000221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f0981-b67d-4286-b778-070202075c65 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.031395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd67077-78a9-4e73-a90e-f16e18d67718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.032188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd67077-78a9-4e73-a90e-f16e18d67718 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.032477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd67077-78a9-4e73-a90e-f16e18d67718 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.032740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd67077-78a9-4e73-a90e-f16e18d67718 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.042135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:37.043095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd67077-78a9-4e73-a90e-f16e18d67718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4fd67077-78a9-4e73-a90e-f16e18d67718 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:37.051093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:37.052103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:37.052936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 221/444] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:37.053590 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd67077-78a9-4e73-a90e-f16e18d67718 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.064344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:37.104256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f0981-b67d-4286-b778-070202075c65 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.139165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81e827-6998-4e1a-8067-e5f7de255635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.140059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81e827-6998-4e1a-8067-e5f7de255635 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.140364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81e827-6998-4e1a-8067-e5f7de255635 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.140641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81e827-6998-4e1a-8067-e5f7de255635 None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.141807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81e827-6998-4e1a-8067-e5f7de255635 None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.143259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81e827-6998-4e1a-8067-e5f7de255635 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.149906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 224/445] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:36 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.149906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81e827-6998-4e1a-8067-e5f7de255635 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.157697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd67077-78a9-4e73-a90e-f16e18d67718 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.191886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d082f5-fef9-497c-83f2-17b50425ed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.192806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d082f5-fef9-497c-83f2-17b50425ed6b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.193179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d082f5-fef9-497c-83f2-17b50425ed6b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.193462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d082f5-fef9-497c-83f2-17b50425ed6b None admin] PATH_INFO: `/v3/projects/e44154970e4b4efda8f5cd260edbc63e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.194719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d082f5-fef9-497c-83f2-17b50425ed6b None admin] RBAC: Authorizing `identity:get_project(project_id=e44154970e4b4efda8f5cd260edbc63e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.196232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d082f5-fef9-497c-83f2-17b50425ed6b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.197943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 222/446] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/projects/e44154970e4b4efda8f5cd260edbc63e => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.198702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d082f5-fef9-497c-83f2-17b50425ed6b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.247368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81e827-6998-4e1a-8067-e5f7de255635 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.256375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:37.275937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:37.281481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.282154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.282438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.282715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.283484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:37.283800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.285207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.293323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 225/447] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.294107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.299336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d082f5-fef9-497c-83f2-17b50425ed6b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.333894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a181aa-b512-48ff-9d6d-0414148d8e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.334749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a181aa-b512-48ff-9d6d-0414148d8e08 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.335033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a181aa-b512-48ff-9d6d-0414148d8e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.335327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a181aa-b512-48ff-9d6d-0414148d8e08 None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.336521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a181aa-b512-48ff-9d6d-0414148d8e08 None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.337968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a181aa-b512-48ff-9d6d-0414148d8e08 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.339593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 223/448] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.340343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a181aa-b512-48ff-9d6d-0414148d8e08 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.401609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bfe32-1702-494e-88fe-b8597cdcc9f6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.434465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c8667-2e3e-4bd5-afcb-0fbb4bb15668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.435344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c8667-2e3e-4bd5-afcb-0fbb4bb15668 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.435621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c8667-2e3e-4bd5-afcb-0fbb4bb15668 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.435893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c8667-2e3e-4bd5-afcb-0fbb4bb15668 None admin] PATH_INFO: `/v3/groups/fca0623470de40d3b2f9174e1b006f72` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.436822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a181aa-b512-48ff-9d6d-0414148d8e08 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.437021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c8667-2e3e-4bd5-afcb-0fbb4bb15668 None admin] RBAC: Authorizing `identity:get_group(group_id=fca0623470de40d3b2f9174e1b006f72)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.438488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c8667-2e3e-4bd5-afcb-0fbb4bb15668 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.440293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 226/449] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/groups/fca0623470de40d3b2f9174e1b006f72 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.440870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c8667-2e3e-4bd5-afcb-0fbb4bb15668 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.470032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978fc64-90ce-445d-935b-36c1d1aad141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.470685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978fc64-90ce-445d-935b-36c1d1aad141 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.470972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978fc64-90ce-445d-935b-36c1d1aad141 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.471285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978fc64-90ce-445d-935b-36c1d1aad141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.479129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:37.488379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:37.541723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978fc64-90ce-445d-935b-36c1d1aad141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.542993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c8667-2e3e-4bd5-afcb-0fbb4bb15668 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.543457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978fc64-90ce-445d-935b-36c1d1aad141 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.577024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.577688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.577969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.578243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.579078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] RBAC: Adding query filter params (group.id=fca0623470de40d3b2f9174e1b006f72, role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=e44154970e4b4efda8f5cd260edbc63e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:37.579428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.580874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.587160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 227/450] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/role_assignments?group.id=fca0623470de40d3b2f9174e1b006f72&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=798b7bcb53684ab49ae899f8902f444a => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.588319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.671933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 224/451] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:37.672931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978fc64-90ce-445d-935b-36c1d1aad141 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.706535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3adda4-ebc4-483c-a91d-51abf6d75904 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.708570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:37.725610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:37.745457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.746129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.746415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.746691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.747904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] RBAC: Adding query filter params (group.id=fca0623470de40d3b2f9174e1b006f72, role.id=af1e89bc75e74c9f819959d898d086a5, scope.project.id=e44154970e4b4efda8f5cd260edbc63e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:37.748363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.749679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.756205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 228/452] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/role_assignments?group.id=fca0623470de40d3b2f9174e1b006f72&scope.project.id=e44154970e4b4efda8f5cd260edbc63e&role.id=af1e89bc75e74c9f819959d898d086a5 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.794000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978fc64-90ce-445d-935b-36c1d1aad141 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.834352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed705278-9fd9-43af-8aa8-842ad90f3138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.835284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705278-9fd9-43af-8aa8-842ad90f3138 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.835565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705278-9fd9-43af-8aa8-842ad90f3138 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.835842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705278-9fd9-43af-8aa8-842ad90f3138 None admin] PATH_INFO: `/v3/projects/44e27258dd6f419b9b00f5d07551d7c9` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.837385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed705278-9fd9-43af-8aa8-842ad90f3138 None admin] RBAC: Authorizing `identity:get_project(project_id=44e27258dd6f419b9b00f5d07551d7c9)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.838852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed705278-9fd9-43af-8aa8-842ad90f3138 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.840618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 225/453] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/projects/44e27258dd6f419b9b00f5d07551d7c9 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.846576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:37.952909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc93a9-4dae-4db6-a0db-c3f4171368f7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:37.986848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a0261-300e-49bd-881c-a6b1e2a16aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:37.987745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0261-300e-49bd-881c-a6b1e2a16aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:37.988022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0261-300e-49bd-881c-a6b1e2a16aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:37.988276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0261-300e-49bd-881c-a6b1e2a16aa3 None admin] PATH_INFO: `/v3/groups/00fcad38810e4e77840c04b0ac44441c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:37.989491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a0261-300e-49bd-881c-a6b1e2a16aa3 None admin] RBAC: Authorizing `identity:get_group(group_id=00fcad38810e4e77840c04b0ac44441c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:37.991007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a0261-300e-49bd-881c-a6b1e2a16aa3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:37.992661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 229/454] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/groups/00fcad38810e4e77840c04b0ac44441c => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:37.999085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed705278-9fd9-43af-8aa8-842ad90f3138 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:38.097601 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed705278-9fd9-43af-8aa8-842ad90f3138 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:38.110871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:38.133149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720ee16-e7fe-4e90-96e0-13cd0687aa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:38.133806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720ee16-e7fe-4e90-96e0-13cd0687aa8c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:38.134087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720ee16-e7fe-4e90-96e0-13cd0687aa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:38.134361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720ee16-e7fe-4e90-96e0-13cd0687aa8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:38.135190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720ee16-e7fe-4e90-96e0-13cd0687aa8c None admin] RBAC: Adding query filter params (group.id=00fcad38810e4e77840c04b0ac44441c, role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=44e27258dd6f419b9b00f5d07551d7c9) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:38.135527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720ee16-e7fe-4e90-96e0-13cd0687aa8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:38.136976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720ee16-e7fe-4e90-96e0-13cd0687aa8c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:38.143017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 226/455] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 08:07:37 2022] GET /identity/v3/role_assignments?group.id=00fcad38810e4e77840c04b0ac44441c&scope.project.id=44e27258dd6f419b9b00f5d07551d7c9&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:40.755796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b84b1-9ab9-4e34-b1e6-6ea6231d254c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:40.756335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b84b1-9ab9-4e34-b1e6-6ea6231d254c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:40.756335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b84b1-9ab9-4e34-b1e6-6ea6231d254c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:40.758172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 230/456] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:40.763627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d28ae-a804-4bd6-a31d-6370c2d0f618 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:40.764041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d28ae-a804-4bd6-a31d-6370c2d0f618 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:40.764210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d28ae-a804-4bd6-a31d-6370c2d0f618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:40.788223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-323d28ae-a804-4bd6-a31d-6370c2d0f618 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:40.789407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-323d28ae-a804-4bd6-a31d-6370c2d0f618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:40.881515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 227/457] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:40.914006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b7a38-d216-433a-8269-7066f9ca959d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:40.914306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b7a38-d216-433a-8269-7066f9ca959d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:40.914553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b7a38-d216-433a-8269-7066f9ca959d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:40.916196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 231/458] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:40.918741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d28ae-a804-4bd6-a31d-6370c2d0f618 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:40.931696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ccedc-6e37-45c7-ac95-4d743d40b335 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:40.931977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ccedc-6e37-45c7-ac95-4d743d40b335 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:40.932255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ccedc-6e37-45c7-ac95-4d743d40b335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:40.956565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-556ccedc-6e37-45c7-ac95-4d743d40b335 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:40.957936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-556ccedc-6e37-45c7-ac95-4d743d40b335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:41.010818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:41.041956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d28ae-a804-4bd6-a31d-6370c2d0f618 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.049718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:41.060697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 232/459] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:41.068822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428687b-e5b1-479d-a481-1639cb38529e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.069321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428687b-e5b1-479d-a481-1639cb38529e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.069764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428687b-e5b1-479d-a481-1639cb38529e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.071562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 233/460] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:41.076467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cc2a2-5c66-4edf-840f-3f60ed635615 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.076935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cc2a2-5c66-4edf-840f-3f60ed635615 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.077398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cc2a2-5c66-4edf-840f-3f60ed635615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.087587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.088714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.090192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.091608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.101977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b95cc2a2-5c66-4edf-840f-3f60ed635615 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:41.103091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:41.103783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b95cc2a2-5c66-4edf-840f-3f60ed635615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:41.104631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:41.123010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:41.123933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:41.125363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:41.125363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:41.125363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:41.125363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:41.125363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 228/461] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:41.125643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.178565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 234/462] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:41.179929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cc2a2-5c66-4edf-840f-3f60ed635615 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.233620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b1ca5-1e31-4aea-84c5-9e2b9cdcb6c1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.267631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014fc82-8f67-4169-86c0-2dafc8b82365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.268355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014fc82-8f67-4169-86c0-2dafc8b82365 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.268636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014fc82-8f67-4169-86c0-2dafc8b82365 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.268908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014fc82-8f67-4169-86c0-2dafc8b82365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.279910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014fc82-8f67-4169-86c0-2dafc8b82365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:41.281613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014fc82-8f67-4169-86c0-2dafc8b82365 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:41.282632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:41.291297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cc2a2-5c66-4edf-840f-3f60ed635615 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a014fc82-8f67-4169-86c0-2dafc8b82365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:41.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:41.299686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:41.300524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:41.300524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:41.300524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:41.300524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:41.300524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 229/463] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:41.301557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752b383-82e9-4498-94e1-a8a6aea09cb6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.301557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752b383-82e9-4498-94e1-a8a6aea09cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.301802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752b383-82e9-4498-94e1-a8a6aea09cb6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.307657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 230/464] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:41.307657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752b383-82e9-4498-94e1-a8a6aea09cb6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.318165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:41.334538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.335221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.335552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.335998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.336542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:41.336939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:41.338240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:41.350916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 235/465] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:41.351749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.412208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752b383-82e9-4498-94e1-a8a6aea09cb6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.447326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56641069-5d9a-4bc6-9527-446bd35d74d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.448124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641069-5d9a-4bc6-9527-446bd35d74d3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.448406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641069-5d9a-4bc6-9527-446bd35d74d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.448679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641069-5d9a-4bc6-9527-446bd35d74d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.459410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56641069-5d9a-4bc6-9527-446bd35d74d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:41.460882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56641069-5d9a-4bc6-9527-446bd35d74d3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:41.464445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aef553-733f-41b9-a71b-220985bf2713 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-56641069-5d9a-4bc6-9527-446bd35d74d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:41.478955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:41.479969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:41.480957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:41.480957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:41.480957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:41.480957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:41.486204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 231/466] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:41.486438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51ad8e-ba07-41e0-b5a8-3e2325780d26 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.486726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51ad8e-ba07-41e0-b5a8-3e2325780d26 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.486995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51ad8e-ba07-41e0-b5a8-3e2325780d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.505405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4c51ad8e-ba07-41e0-b5a8-3e2325780d26 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:41.506612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4c51ad8e-ba07-41e0-b5a8-3e2325780d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:41.513476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.514156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.514436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.514684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.515808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:41.516128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:41.517536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:41.530553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 236/467] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:41.531190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.547831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:41.574362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:41.586201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 232/468] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:41.592389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51ad8e-ba07-41e0-b5a8-3e2325780d26 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.636988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb7c0f-a068-4e11-b1bd-4e25bc73eef8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.670519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a37c5-2e3c-4796-9ca5-6c61859cff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.671180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a37c5-2e3c-4796-9ca5-6c61859cff1d None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.671483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a37c5-2e3c-4796-9ca5-6c61859cff1d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.671757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a37c5-2e3c-4796-9ca5-6c61859cff1d None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.672463 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a37c5-2e3c-4796-9ca5-6c61859cff1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:41.673916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a37c5-2e3c-4796-9ca5-6c61859cff1d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:41.700121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 237/469] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:41 2022] POST /identity/v3/users => generated 248 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:41.700914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a37c5-2e3c-4796-9ca5-6c61859cff1d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.718947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51ad8e-ba07-41e0-b5a8-3e2325780d26 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.779130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.780926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.781397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.782087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:41.782762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.784093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:41.784405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:41.785915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:41.812982 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 233/470] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:41.812982 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.832181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a37c5-2e3c-4796-9ca5-6c61859cff1d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.877235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:41.885016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc154-2c95-4200-b545-f83cf6c1bd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.886020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc154-2c95-4200-b545-f83cf6c1bd99 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.886446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc154-2c95-4200-b545-f83cf6c1bd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.886842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc154-2c95-4200-b545-f83cf6c1bd99 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:41.888296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fc154-2c95-4200-b545-f83cf6c1bd99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:41.890918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fc154-2c95-4200-b545-f83cf6c1bd99 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:41.917835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 238/471] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:41 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:41.923357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc154-2c95-4200-b545-f83cf6c1bd99 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:41.933517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba0b19-7202-4479-a58c-325323a88e07 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:41.985059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede6ca4-d24a-40d3-b3b0-1207099ff7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:41.985884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede6ca4-d24a-40d3-b3b0-1207099ff7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:41.986168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede6ca4-d24a-40d3-b3b0-1207099ff7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:41.986443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede6ca4-d24a-40d3-b3b0-1207099ff7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.000303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede6ca4-d24a-40d3-b3b0-1207099ff7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:42.001793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede6ca4-d24a-40d3-b3b0-1207099ff7b9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-bede6ca4-d24a-40d3-b3b0-1207099ff7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:42.024563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:42.025494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:42.026328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:42.026328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:42.026328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:42.026328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:42.027030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 234/472] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:42.032350 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03429c0d-a7cf-4d10-8b37-a983c90e63c1 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:42.032613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03429c0d-a7cf-4d10-8b37-a983c90e63c1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:42.035384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03429c0d-a7cf-4d10-8b37-a983c90e63c1 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.037339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 235/473] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:42.038791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03429c0d-a7cf-4d10-8b37-a983c90e63c1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:42.081146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc154-2c95-4200-b545-f83cf6c1bd99 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:42.109387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:42.120810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312856e-18aa-4384-bde7-8ba98a0c4d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:42.121492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312856e-18aa-4384-bde7-8ba98a0c4d33 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:42.122659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312856e-18aa-4384-bde7-8ba98a0c4d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:42.122659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312856e-18aa-4384-bde7-8ba98a0c4d33 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.123051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312856e-18aa-4384-bde7-8ba98a0c4d33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:42.124575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312856e-18aa-4384-bde7-8ba98a0c4d33 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:42.151234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 239/474] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:41 2022] POST /identity/v3/users => generated 246 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:42.151234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86b121-8765-4ebb-bd88-23bfe71e9f0d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:42.151234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86b121-8765-4ebb-bd88-23bfe71e9f0d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:42.151234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86b121-8765-4ebb-bd88-23bfe71e9f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.170546 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2e86b121-8765-4ebb-bd88-23bfe71e9f0d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:42.171993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2e86b121-8765-4ebb-bd88-23bfe71e9f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:42.177445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03429c0d-a7cf-4d10-8b37-a983c90e63c1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:42.180715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:42.228516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:42.229719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:42.229997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:42.230268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.231508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:42.232118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:42.233810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:42.251303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 236/475] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:42.257083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:42.261462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 240/476] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:42.288179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86b121-8765-4ebb-bd88-23bfe71e9f0d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:42.370843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dccea-0ff3-47de-b4a2-0dc46e9ddecd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:42.399359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86b121-8765-4ebb-bd88-23bfe71e9f0d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:42.407665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5147cb-f2d6-453a-85a2-7dca3390d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:42.408666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5147cb-f2d6-453a-85a2-7dca3390d276 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:42.409138 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5147cb-f2d6-453a-85a2-7dca3390d276 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:42.409621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5147cb-f2d6-453a-85a2-7dca3390d276 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.410511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5147cb-f2d6-453a-85a2-7dca3390d276 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:42.412208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5147cb-f2d6-453a-85a2-7dca3390d276 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:42.414357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:42.436216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 237/477] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:42 2022] POST /identity/v3/users => generated 249 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:42.438987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:42.439770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:42.440053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:42.440326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.441096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:42.442583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:42.455308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 08:07:42.455308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 08:07:42.455308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [parameters: {'id': 'c734ab80cf074506b4ad5607e43a049f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 08:07:42.455308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jun 23 08:07:42.473615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 08:07:42.474671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 08:07:42.475649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 08:07:42.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 08:07:42.477393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 08:07:42.478193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'id': 'c734ab80cf074506b4ad5607e43a049f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:42.478961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 08:07:42.479836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:42.480663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 241/478] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 08:07:42 2022] POST /identity/v3/roles => generated 144 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:42.487709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5147cb-f2d6-453a-85a2-7dca3390d276 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:42.516297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:42.617342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5147cb-f2d6-453a-85a2-7dca3390d276 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:42.653338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e583ef-eeb3-4fe7-9463-6c27ab20994f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:42.654456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e583ef-eeb3-4fe7-9463-6c27ab20994f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:42.654887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e583ef-eeb3-4fe7-9463-6c27ab20994f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:42.655372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e583ef-eeb3-4fe7-9463-6c27ab20994f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.666747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e583ef-eeb3-4fe7-9463-6c27ab20994f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-34e583ef-eeb3-4fe7-9463-6c27ab20994f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:42.679338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:42.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 08:07:42.682545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:42.686491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 238/479] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:07:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:42.694749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:42.816806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969376b-7b6c-458e-b077-d7359512dec6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:42.856438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d7005-78df-4986-9496-1e0855b4ea19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:42.857487 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d7005-78df-4986-9496-1e0855b4ea19 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:42.857930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d7005-78df-4986-9496-1e0855b4ea19 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:42.858338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d7005-78df-4986-9496-1e0855b4ea19 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:42.859262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d7005-78df-4986-9496-1e0855b4ea19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:42.859739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d7005-78df-4986-9496-1e0855b4ea19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:42.861267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d7005-78df-4986-9496-1e0855b4ea19 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:42.871045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 242/480] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:44.245394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65e64-b52e-41a1-9d30-fadf1612a7c8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:44.245934 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65e64-b52e-41a1-9d30-fadf1612a7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:44.245934 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65e64-b52e-41a1-9d30-fadf1612a7c8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:44.247909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 239/481] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:44.252981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d131e4f-6cdb-4071-96df-286a3f0d82ce None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:44.253377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d131e4f-6cdb-4071-96df-286a3f0d82ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:44.253536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d131e4f-6cdb-4071-96df-286a3f0d82ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:44.270384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2d131e4f-6cdb-4071-96df-286a3f0d82ce None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:44.271721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2d131e4f-6cdb-4071-96df-286a3f0d82ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:44.312493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:44.365859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 243/482] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:44.401460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff65e64-b52e-41a1-9d30-fadf1612a7c8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:44.492224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:44.518596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff65e64-b52e-41a1-9d30-fadf1612a7c8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:44.557823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f225e-d74e-439c-a016-92e9a9cb7280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:44.558640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f225e-d74e-439c-a016-92e9a9cb7280 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:44.558916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f225e-d74e-439c-a016-92e9a9cb7280 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:44.559188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f225e-d74e-439c-a016-92e9a9cb7280 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:44.580445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f225e-d74e-439c-a016-92e9a9cb7280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3e9f225e-d74e-439c-a016-92e9a9cb7280 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:44.588262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:44.589224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:44.590003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 240/483] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:44.595739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d131e4f-6cdb-4071-96df-286a3f0d82ce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:44.630529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eda59-5578-4353-af0e-b39d0f0b64e6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:44.630840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eda59-5578-4353-af0e-b39d0f0b64e6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:44.631087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eda59-5578-4353-af0e-b39d0f0b64e6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:44.632771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 241/484] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:07:44.638351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512d543-76c6-4e61-92b7-37dfda79c4f4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:44.638629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512d543-76c6-4e61-92b7-37dfda79c4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:44.638901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512d543-76c6-4e61-92b7-37dfda79c4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:44.666049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8512d543-76c6-4e61-92b7-37dfda79c4f4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:44.667272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8512d543-76c6-4e61-92b7-37dfda79c4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:44.726203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d131e4f-6cdb-4071-96df-286a3f0d82ce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:44.769043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:44.770060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:44.770538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:44.771058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:44.772050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:44.772551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:44.774166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:44.790303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 244/485] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:44.795084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:44.811726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 242/486] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:44.843044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:44.858129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b37910-58f0-4b0d-9c91-894261b19666 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:44.858829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b37910-58f0-4b0d-9c91-894261b19666 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:44.859397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b37910-58f0-4b0d-9c91-894261b19666 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:44.861291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 243/487] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:44.862250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b37910-58f0-4b0d-9c91-894261b19666 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:44.910276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1da29-0964-4bfe-99fe-a822e84531b5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:44.940999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:44.948658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca9897-ef87-47ec-b724-457b18bbebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:44.951822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca9897-ef87-47ec-b724-457b18bbebae None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:44.952107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca9897-ef87-47ec-b724-457b18bbebae None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:44.952379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca9897-ef87-47ec-b724-457b18bbebae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:44.973212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca9897-ef87-47ec-b724-457b18bbebae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:44.976639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca9897-ef87-47ec-b724-457b18bbebae None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:44.981858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b37910-58f0-4b0d-9c91-894261b19666 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2fca9897-ef87-47ec-b724-457b18bbebae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:45.001924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:45.002815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:45.004038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:45.004038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:45.004038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:45.004038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:45.004038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 245/488] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:45.005255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e591d4-910d-4fea-8999-1e06d33eec48 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.005539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e591d4-910d-4fea-8999-1e06d33eec48 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.005815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e591d4-910d-4fea-8999-1e06d33eec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.027402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-58e591d4-910d-4fea-8999-1e06d33eec48 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:45.027571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a849c6-b9c5-4332-b769-dfef99d2a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.028288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a849c6-b9c5-4332-b769-dfef99d2a7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.028566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a849c6-b9c5-4332-b769-dfef99d2a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.028843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a849c6-b9c5-4332-b769-dfef99d2a7e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.029640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58e591d4-910d-4fea-8999-1e06d33eec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:45.041201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a849c6-b9c5-4332-b769-dfef99d2a7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-70a849c6-b9c5-4332-b769-dfef99d2a7e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:45.049697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:45.050586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:45.051442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:45.056015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 244/489] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:45.056976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a849c6-b9c5-4332-b769-dfef99d2a7e0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.103937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 246/490] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:45.105099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e591d4-910d-4fea-8999-1e06d33eec48 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.127997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:45.199943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a849c6-b9c5-4332-b769-dfef99d2a7e0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.232677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e591d4-910d-4fea-8999-1e06d33eec48 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.239647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:45.251760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.252526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.252731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.252976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.253708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:45.254038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:45.256141 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:45.273054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.274406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.274882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.275376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.275595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 245/491] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:45.276555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:45.276954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.277257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:45.278962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:45.288899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 247/492] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:45.290031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.375985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:45.402001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6be53-d649-4f7b-b77f-ef63330070cc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.414795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b929bb-39b5-438c-a5c3-0b9ce81dfd09 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.435953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.436771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.437049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.437318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.445116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:45.446575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:45.451885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964dc2-a7c8-4ae2-9a01-7c8dc4447a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.452934 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964dc2-a7c8-4ae2-9a01-7c8dc4447a5e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.453385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964dc2-a7c8-4ae2-9a01-7c8dc4447a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.453822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964dc2-a7c8-4ae2-9a01-7c8dc4447a5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:45.462590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:45.463550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:45.464470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 248/493] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:45.465112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.467522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed964dc2-a7c8-4ae2-9a01-7c8dc4447a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ed964dc2-a7c8-4ae2-9a01-7c8dc4447a5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:45.475684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:45.476581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:45.477373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:45.479501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 246/494] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:45.481554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a6147-e6f0-4be9-ae62-433470eff34b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.481842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a6147-e6f0-4be9-ae62-433470eff34b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.482454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a6147-e6f0-4be9-ae62-433470eff34b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.486299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 247/495] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:45.486299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a6147-e6f0-4be9-ae62-433470eff34b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.540893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:45.585784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e85a2-c782-4c0f-bdfe-53469748eef8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.598561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a6147-e6f0-4be9-ae62-433470eff34b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.622257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.623062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.623355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.623635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.625660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:45.635280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b7a2b-9418-4bfc-990c-322b18474177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.635895 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:45.636033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b7a2b-9418-4bfc-990c-322b18474177 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.636203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b7a2b-9418-4bfc-990c-322b18474177 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.636471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b7a2b-9418-4bfc-990c-322b18474177 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.637306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b7a2b-9418-4bfc-990c-322b18474177 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:45.637395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:45.637613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b7a2b-9418-4bfc-990c-322b18474177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:45.639032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b7a2b-9418-4bfc-990c-322b18474177 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:45.654683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:45.655617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:45.656499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:45.656499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:45.656499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:45.656499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:45.656499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 249/496] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:45.657093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.677508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 248/497] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:45.679093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbd320-21e2-49b6-b5ff-17e61c6dea6b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.679430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbd320-21e2-49b6-b5ff-17e61c6dea6b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.679710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbd320-21e2-49b6-b5ff-17e61c6dea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.695991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-acbbd320-21e2-49b6-b5ff-17e61c6dea6b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:45.697172 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-acbbd320-21e2-49b6-b5ff-17e61c6dea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:45.761296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22f523-7a1b-417f-86c0-595dc71b56f1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.765537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 249/498] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:45.766843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a973978-9b33-420f-81e5-1e3c5feaafda None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.767129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a973978-9b33-420f-81e5-1e3c5feaafda None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.767433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a973978-9b33-420f-81e5-1e3c5feaafda None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.768722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 250/499] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:45.769394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a973978-9b33-420f-81e5-1e3c5feaafda None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.807689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.808270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.808548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.808815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.809539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:45.809839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:45.811249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:45.815868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:45.819699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 250/500] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:45.819699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:45.868936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:45.881314 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a973978-9b33-420f-81e5-1e3c5feaafda None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.915275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3226ea-bafe-413a-87a2-b66e3e079461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.915933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3226ea-bafe-413a-87a2-b66e3e079461 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.916205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3226ea-bafe-413a-87a2-b66e3e079461 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.916476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3226ea-bafe-413a-87a2-b66e3e079461 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.917187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3226ea-bafe-413a-87a2-b66e3e079461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:45.917483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3226ea-bafe-413a-87a2-b66e3e079461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:45.918864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3226ea-bafe-413a-87a2-b66e3e079461 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:45.921065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fc338-d24d-4866-8fab-1a28ace69a62 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:45.929727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 251/501] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:45.931091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c2a9-add2-41f0-9c17-20b799780638 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.931426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c2a9-add2-41f0-9c17-20b799780638 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.931686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c2a9-add2-41f0-9c17-20b799780638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:45.948604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-52f8c2a9-add2-41f0-9c17-20b799780638 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:45.950309 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-52f8c2a9-add2-41f0-9c17-20b799780638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:45.954823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ce317-7d63-460f-9bcf-66e2a184eb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:45.955516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce317-7d63-460f-9bcf-66e2a184eb42 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:45.959793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce317-7d63-460f-9bcf-66e2a184eb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:45.960081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce317-7d63-460f-9bcf-66e2a184eb42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.022463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 252/502] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:46.023352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8c2a9-add2-41f0-9c17-20b799780638 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.031011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ce317-7d63-460f-9bcf-66e2a184eb42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.032532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ce317-7d63-460f-9bcf-66e2a184eb42 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.056313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.084282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.127870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8c2a9-add2-41f0-9c17-20b799780638 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.137953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 251/503] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:46.137953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ce317-7d63-460f-9bcf-66e2a184eb42 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.162959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68ca49-fa0e-45da-8a59-9ef9e4782bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.163795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68ca49-fa0e-45da-8a59-9ef9e4782bdc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.164076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68ca49-fa0e-45da-8a59-9ef9e4782bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.164344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68ca49-fa0e-45da-8a59-9ef9e4782bdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.174917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68ca49-fa0e-45da-8a59-9ef9e4782bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-5f68ca49-fa0e-45da-8a59-9ef9e4782bdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:46.183366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:46.184432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:46.185270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 253/504] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:46.185933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68ca49-fa0e-45da-8a59-9ef9e4782bdc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.244084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ce317-7d63-460f-9bcf-66e2a184eb42 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.274230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.277477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.278280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.278554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.278834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.286406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68ca49-fa0e-45da-8a59-9ef9e4782bdc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.288884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.290347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.298483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:46.308699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:46.309570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:46.310406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:46.310406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:46.310406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:46.310406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:46.310406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 252/505] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:46.310889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.326660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.327493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.327776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.328047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.336176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.337565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:46.353221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:46.354118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 254/506] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:46.355198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.412385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e60ab-a689-47a3-8719-7871e1b7f346 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.445913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.446705 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.446986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.447294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.456696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ec8c8-089b-4533-a88d-d8d67f6bff3b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.457064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.458574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:46.476259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:46.477121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:46.477965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:46.477965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:46.477965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:46.477965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:46.477965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 253/507] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:46.478230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.494577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.495396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.495678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.495952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.507506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.508979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.517155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:46.528320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:46.529339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:46.530257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:46.530257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:46.530257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:46.530257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:46.532011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 255/508] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:46.536667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.539807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.586557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e7e77-edf3-4f09-b915-5a5d38f5379e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.622482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.623148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.623470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.623751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.624473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:46.624776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.626175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.634475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 254/509] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:46.635296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.635858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3773c57-b0fe-48e2-a5f5-18656a4cf6b1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.670842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.671533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.671811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.672084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.672808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:46.673108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.674518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.686259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 256/510] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:46.687031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.744816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dacd6f-85e1-493b-b488-fb1762004a73 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.759253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.761397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.787179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.788100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.788711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.789054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.789965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:46.790339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.792296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.803081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822ded1-cb25-427b-8e4a-f932c1b88d7d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.836662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.836874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 255/511] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:46.837389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.837730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.837868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.838115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.838909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:46.839330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.840802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:46.853209 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 257/512] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:46.854218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:46.946291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de660c77-5960-4ca4-b5ed-af496f211a0b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.959580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0982210-16a9-433c-9f87-64b319b00a12 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:46.983181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:46.983380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:46.984035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:46.984321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:46.984597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:46.985497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:46.985779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:46.987430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.000112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 256/513] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:47.000255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.000926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.001025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.001336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.001611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.014332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.015822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:47.032971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:47.033940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:47.034770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:47.034770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:47.034770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:47.034770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:47.034770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 258/514] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:47.035449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.039318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:47.107571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aac62-c677-4bc3-84f8-0c3eddc3f3a6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.136864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0ba63-1be5-462b-941c-caa37cb021a5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.140392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.141189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.141473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.141745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.149166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.150638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:47.164854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:47.165812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 257/515] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:47.166911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.173320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e04157-18e7-4cc4-adbd-75aeb4d5b1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.173971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04157-18e7-4cc4-adbd-75aeb4d5b1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.174250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04157-18e7-4cc4-adbd-75aeb4d5b1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.174520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04157-18e7-4cc4-adbd-75aeb4d5b1e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.216204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:47.244631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e04157-18e7-4cc4-adbd-75aeb4d5b1e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.246402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e04157-18e7-4cc4-adbd-75aeb4d5b1e0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.258695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:47.267988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3b382-07b5-4208-b1f9-a1816c95e4cc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.303183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada1305-b9eb-485b-a499-6dc98e6ebc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.303909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1305-b9eb-485b-a499-6dc98e6ebc4f None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.304188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1305-b9eb-485b-a499-6dc98e6ebc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.304460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1305-b9eb-485b-a499-6dc98e6ebc4f None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.305163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada1305-b9eb-485b-a499-6dc98e6ebc4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.306628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada1305-b9eb-485b-a499-6dc98e6ebc4f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.326888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 258/516] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:07:47 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:07:47.327730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada1305-b9eb-485b-a499-6dc98e6ebc4f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.363867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 259/517] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:47.364716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e04157-18e7-4cc4-adbd-75aeb4d5b1e0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.444388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:47.452862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada1305-b9eb-485b-a499-6dc98e6ebc4f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.475752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e04157-18e7-4cc4-adbd-75aeb4d5b1e0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.499019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.500333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.500616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.501560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.509446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.511299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.527277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.527948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.528225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.528495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.529289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:47.529574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:47.529776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:47.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:47.531468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:47.531468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:47.531468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:47.531468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:47.531468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:47.531468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:47.531468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:47.531468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:47.531911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.531979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 259/518] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:47.531979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.550218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 260/519] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:47.551070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.551598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:47.648201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb2a5d-f5c4-4bbb-a7b4-a0f606f1f0fc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.657044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38187a0d-af91-4bd3-97d4-430f3ad11921 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.685694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.686367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.686645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.686911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.687775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:47.688164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.689538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.690909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:47.698706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.699593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.699883 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.700158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.711490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 260/520] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:47.711490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.711740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.713231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:47.731671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:47.732525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:47.733359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:47.733359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:47.733359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:47.733359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:47.733359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 261/521] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:47.734057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.780239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:47.821372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e2b14-5f04-4dc5-b80d-a0cd2856f8e3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.844549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e6026-4286-4eca-8491-dc564eba6f66 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:47.856364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.857028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.857304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.857572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.858355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:47.858661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.860160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.877925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 261/522] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:47.878737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.888354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:47.889435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:47.889720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:47.889995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:47.899716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:47.901089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:47.918904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:47.919887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 262/523] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:47.921232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:47.970092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:47.993746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd3b21-6cc3-44b7-9ec0-a5bb9a5334f8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:48.020307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:48.034187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3099-dab6-421c-a2d8-8e3a5b96bfbf None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:48.043657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb697cae-a2df-4bcf-8ae7-6388f88cdfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:48.044453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb697cae-a2df-4bcf-8ae7-6388f88cdfaa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:48.044625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb697cae-a2df-4bcf-8ae7-6388f88cdfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:48.044901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb697cae-a2df-4bcf-8ae7-6388f88cdfaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:48.077665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:48.078339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:48.078617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:48.078884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:48.079714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:48.080015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:48.081389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:48.094346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 263/524] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:48.095080 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:48.126921 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb697cae-a2df-4bcf-8ae7-6388f88cdfaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:48.128441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb697cae-a2df-4bcf-8ae7-6388f88cdfaa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:48.204556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b62bb4-a150-44c9-bd35-12e9352c2129 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:48.207537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:48.236469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 262/525] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:48.236469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb697cae-a2df-4bcf-8ae7-6388f88cdfaa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:48.250476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:48.251131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:48.251433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:48.251702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:48.252532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=d8cbcc9295074710aeacb35e4e1a4348) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:48.252839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:48.254272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:48.259875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 264/526] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:48 2022] GET /identity/v3/role_assignments?user.id=d8cbcc9295074710aeacb35e4e1a4348&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:48.260636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:48.280533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:48.354822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb697cae-a2df-4bcf-8ae7-6388f88cdfaa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:48.396370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:48.397056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:48.397340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:48.397616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:48.398460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:48.398767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:48.400280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:48.407466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4869a-503c-4c29-8f82-67a842b22e25 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:48.425164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 263/527] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 23 08:07:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:48.426576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:48.451520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:48.452355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:48.452639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:48.452915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:48.467329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:48.467958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:48.487365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:48.488252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:48.489085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:48.489085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:48.489085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:48.489085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:48.489085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:48.489085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:48.489085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:48.489085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:48.489619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 265/528] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:48.490283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:48.534544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:48.543051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:48.563583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ef3dc-8738-4d73-a1da-5a01623cbec5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:48.605513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:48.606504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:48.607063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:48.607369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:48.619850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:48.622374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:48.624774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c501eb2-5535-420e-9d12-bd5786a6858e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:48.647453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:48.649136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:48.650584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:48.650584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:48.650584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:48.650584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:48.650584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 264/529] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:48.650584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:48.671688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f024-a7f1-428d-8bcc-ba564736ee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:48.672494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f024-a7f1-428d-8bcc-ba564736ee2f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:48.672788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f024-a7f1-428d-8bcc-ba564736ee2f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:48.673341 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f024-a7f1-428d-8bcc-ba564736ee2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:48.758180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913f024-a7f1-428d-8bcc-ba564736ee2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:48.759971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913f024-a7f1-428d-8bcc-ba564736ee2f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:48.770990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7c7e8-b1d3-47aa-8c62-bd425f7ac4c8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:48.793136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:48.812644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:48.813955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:48.814733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:48.815167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:48.816368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:48.816847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:48.819531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:48.831570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:48.837564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 265/530] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:48.838387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:48.887407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 266/531] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:48.888265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f024-a7f1-428d-8bcc-ba564736ee2f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:48.970991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62897b6d-b620-43e3-973a-f0cda66abe4b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.007438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.008552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.009062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.009567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.010511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:49.011038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:49.012721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:49.021763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f024-a7f1-428d-8bcc-ba564736ee2f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.030504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 266/532] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:49.031803 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.056473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:49.068459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.069154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.069444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.069746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.070862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=419321fbd62c4d95a39cfd498b4a97d7) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:49.072091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:49.073554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:49.080537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 267/533] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:48 2022] GET /identity/v3/role_assignments?user.id=419321fbd62c4d95a39cfd498b4a97d7&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:49.081350 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.097669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:49.149836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dbe5d-5127-4a6c-98f2-10930a56229c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.188270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.189444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.189940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.190422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.202555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:49.204465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:49.223811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7115fb4-f986-4d7e-8d77-e4ea31ac60c2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:49.224385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:49.225225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:49.226053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:49.226053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:49.226053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:49.226053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:49.228400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 267/534] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:49.245220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.277325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.278562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.279039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.279526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.295870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:49.298511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:49.330145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:49.331049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:49.331919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 268/535] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:49.338432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.394628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f755e8-3fa2-4ca2-ad24-e5b90f0840d5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.405070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:49.405231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:49.440861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d3c1d-2669-45d3-bfbb-a1b1cd8aab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.441828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d3c1d-2669-45d3-bfbb-a1b1cd8aab2e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.442552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d3c1d-2669-45d3-bfbb-a1b1cd8aab2e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.442974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d3c1d-2669-45d3-bfbb-a1b1cd8aab2e None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.444382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d3c1d-2669-45d3-bfbb-a1b1cd8aab2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:49.445100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d3c1d-2669-45d3-bfbb-a1b1cd8aab2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:49.447049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d3c1d-2669-45d3-bfbb-a1b1cd8aab2e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:49.462630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 268/536] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:49.463127 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eccb55-ad4e-45f6-ad69-db348408da68 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.464719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f99d91-dbe5-4346-b03d-7b454bc40b87 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.465184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f99d91-dbe5-4346-b03d-7b454bc40b87 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.465661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f99d91-dbe5-4346-b03d-7b454bc40b87 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.471264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 269/537] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:49.472296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f99d91-dbe5-4346-b03d-7b454bc40b87 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.502663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398a93c-87b3-4a16-866d-067e6c7f3aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.503360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a93c-87b3-4a16-866d-067e6c7f3aab None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.503643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a93c-87b3-4a16-866d-067e6c7f3aab None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.504011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a93c-87b3-4a16-866d-067e6c7f3aab None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.504837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398a93c-87b3-4a16-866d-067e6c7f3aab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:49.505181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398a93c-87b3-4a16-866d-067e6c7f3aab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:49.506669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398a93c-87b3-4a16-866d-067e6c7f3aab None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:49.527577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 269/538] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:49.528904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c394eec-042a-40d9-a852-f0cd68d71d2f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.529191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c394eec-042a-40d9-a852-f0cd68d71d2f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.529466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c394eec-042a-40d9-a852-f0cd68d71d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.546394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8c394eec-042a-40d9-a852-f0cd68d71d2f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:49.547616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8c394eec-042a-40d9-a852-f0cd68d71d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:49.585664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f99d91-dbe5-4346-b03d-7b454bc40b87 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.616565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 270/539] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:49.623288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c394eec-042a-40d9-a852-f0cd68d71d2f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.631922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.633006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.633463 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.633909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.641406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:49.643194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:49.667255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:49.671733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:49.672565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:49.673416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:49.673416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:49.673416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:49.673416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:49.673416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:49.673416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:49.673416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:49.673416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:49.675899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 270/540] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:49.677037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.706488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:49.720548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c394eec-042a-40d9-a852-f0cd68d71d2f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.756450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.757108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.757500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.757691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.758734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=96456bfed82d469b888bd495ec46e796) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:49.758861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:49.760305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:49.766330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 271/541] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/role_assignments?user.id=96456bfed82d469b888bd495ec46e796&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:49.767187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.807891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0dd5c-5382-46cf-bff9-c268623a054f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.850734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bf741-efff-4e0e-a4c6-4f746c0991a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:49.851858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bf741-efff-4e0e-a4c6-4f746c0991a7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:49.852321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bf741-efff-4e0e-a4c6-4f746c0991a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:49.852733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bf741-efff-4e0e-a4c6-4f746c0991a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:49.876800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bf741-efff-4e0e-a4c6-4f746c0991a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a18bf741-efff-4e0e-a4c6-4f746c0991a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:49.897066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:49.898233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:49.899233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:49.903183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 271/542] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:49.925224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bf741-efff-4e0e-a4c6-4f746c0991a7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:49.980621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33592b-e8ce-4251-aa2a-b520caff6f86 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:49.992596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:50.029157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:50.029870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.030158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.030429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.031253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:50.031560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:50.033896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:50.049582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 272/543] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:50.056206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:50.063040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:50.078702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bf741-efff-4e0e-a4c6-4f746c0991a7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:50.118609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:50.119690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.120365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.120779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.122449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:50.122907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:50.124604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:50.138022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 272/544] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:50.140645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:50.250809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c1b34-2123-4191-8534-f5fdbe752329 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:50.258704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae49dd2-a4e4-40f1-8e72-910005dd1754 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:50.287573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:50.288704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.289153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.289614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.301300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:50.303306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:50.320028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:50.320617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.320897 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.321174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.322015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=2965b1c8ab52457f8f167acda4ba6844) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:50.322325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:50.322908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:50.323769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:50.324593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:50.324593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:50.324593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:50.324593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:50.324796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:50.330938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 273/545] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:50.332298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:50.335443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:50.337606 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 273/546] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:50 2022] GET /identity/v3/role_assignments?user.id=2965b1c8ab52457f8f167acda4ba6844&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:50.444811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3856386-f269-42e7-a330-9835c1971abb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:50.485766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90026df3-8a6f-4db7-8248-a82d5dfee7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:50.486705 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90026df3-8a6f-4db7-8248-a82d5dfee7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.487139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90026df3-8a6f-4db7-8248-a82d5dfee7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.487588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90026df3-8a6f-4db7-8248-a82d5dfee7e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.488452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90026df3-8a6f-4db7-8248-a82d5dfee7e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:50.488896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90026df3-8a6f-4db7-8248-a82d5dfee7e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:50.490752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90026df3-8a6f-4db7-8248-a82d5dfee7e0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:50.513213 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 274/547] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:50.516355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:50.579703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:50.652189 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd038928-79c5-4b4e-8f9a-c01d238f3a2c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:50.659755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1700ba2-c565-41fb-bbf1-42a9be30dd9f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.660228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1700ba2-c565-41fb-bbf1-42a9be30dd9f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.660644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1700ba2-c565-41fb-bbf1-42a9be30dd9f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.665829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 275/548] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:50.667477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc197d-37ec-4766-83c6-b499d8a09232 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.667924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc197d-37ec-4766-83c6-b499d8a09232 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.668345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc197d-37ec-4766-83c6-b499d8a09232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.694924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:50.695778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.696107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.696381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.697760 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-52dc197d-37ec-4766-83c6-b499d8a09232 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:50.698958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-52dc197d-37ec-4766-83c6-b499d8a09232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:50.709469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:50.710919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:50.725505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:50.726589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:50.728494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 274/549] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:50.756339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:50.791834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:50.794495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 276/550] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:07:50.828782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc197d-37ec-4766-83c6-b499d8a09232 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:50.871201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:50.882610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c959-f61c-4a48-b7c5-b586481bbcd6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:50.924021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:50.925122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:50.925712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:50.926959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:50.928647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:50.929317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:50.931925 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:50.968610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 275/551] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 08:07:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:50.976220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:50.997552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc197d-37ec-4766-83c6-b499d8a09232 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:51.031696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c76a1-2b3f-4356-af2f-b3e78ceba26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:51.032863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c76a1-2b3f-4356-af2f-b3e78ceba26f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.033211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c76a1-2b3f-4356-af2f-b3e78ceba26f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.033552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c76a1-2b3f-4356-af2f-b3e78ceba26f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.051233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c76a1-2b3f-4356-af2f-b3e78ceba26f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-856c76a1-2b3f-4356-af2f-b3e78ceba26f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:51.052225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:51.053486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:51.054680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:51.055830 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 277/552] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:07:51.064178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c76a1-2b3f-4356-af2f-b3e78ceba26f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:51.121197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:51.153571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43d837-f51d-47be-aa88-22423c15b4df None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:51.170585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c76a1-2b3f-4356-af2f-b3e78ceba26f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:51.213965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ad42d-295c-4c3a-8069-378168602e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:51.214178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:51.215012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.215267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ad42d-295c-4c3a-8069-378168602e6b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.215382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.215580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.215994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ad42d-295c-4c3a-8069-378168602e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.216336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:51.216641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:51.216748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ad42d-295c-4c3a-8069-378168602e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.219404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:51.227126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 278/553] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:51.232666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:51.248447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:51.325377 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ad42d-295c-4c3a-8069-378168602e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:51.326922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ad42d-295c-4c3a-8069-378168602e6b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:51.354410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deeb04-2831-4c96-8863-bf8a133922fd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:51.370173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:51.394834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:51.396191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.396682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.397131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.409460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:51.411372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:51.433059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:51.433933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:51.434798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:51.434798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:51.434798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:51.434798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:51.437197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 279/554] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:51.442323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:51.525979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 276/555] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:51.527194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4762d01-8de4-4a5b-8423-e874fd512fc2 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.527501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4762d01-8de4-4a5b-8423-e874fd512fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.527779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4762d01-8de4-4a5b-8423-e874fd512fc2 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.529173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 277/556] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:51.534496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4762d01-8de4-4a5b-8423-e874fd512fc2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:51.589817 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:51.592646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959644c-fda3-4ff8-af09-6f82aae01320 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:51.630233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3f1bc-9a8a-47cb-9096-c9a6a16427bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:51.631331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3f1bc-9a8a-47cb-9096-c9a6a16427bb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.631875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3f1bc-9a8a-47cb-9096-c9a6a16427bb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.632339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3f1bc-9a8a-47cb-9096-c9a6a16427bb None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.633250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3f1bc-9a8a-47cb-9096-c9a6a16427bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:51.633746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3f1bc-9a8a-47cb-9096-c9a6a16427bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:51.635542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3f1bc-9a8a-47cb-9096-c9a6a16427bb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:51.644799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4762d01-8de4-4a5b-8423-e874fd512fc2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:51.653550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 280/557] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:51.655302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735adec-9370-4f3d-80b8-444e9efe35fe None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.655882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735adec-9370-4f3d-80b8-444e9efe35fe None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.656349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735adec-9370-4f3d-80b8-444e9efe35fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.674370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9735adec-9370-4f3d-80b8-444e9efe35fe None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:51.675973 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9735adec-9370-4f3d-80b8-444e9efe35fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:51.681620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:51.682429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.682711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.682984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.694896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:51.696441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:51.701706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:51.717204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:51.718966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:51.720168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:51.720168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:51.720168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:51.720168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:51.723153 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 278/558] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:51.724381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:51.763495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 281/559] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:51.764807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735adec-9370-4f3d-80b8-444e9efe35fe None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:51.827452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:51.830201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb684d-d855-430e-a5e9-99696d8b35ee None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:51.865601 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:51.866582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.867111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.867366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.875259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:51.876953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:51.890178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735adec-9370-4f3d-80b8-444e9efe35fe None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:51.892240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:51.893364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 08:07:51.894599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:51.900686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 279/560] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:51.900686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:51.940445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:51.941038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:51.941365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:51.941645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:51.942558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:51.942869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:51.945102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:51.963199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 282/561] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:51.965159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:51.991468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:52.029911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5cd85-e8c4-4ec9-a967-a4c245bcbd24 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.073292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d9a09-6495-4d93-9c11-3ce7016ab6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.074125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d9a09-6495-4d93-9c11-3ce7016ab6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.074402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d9a09-6495-4d93-9c11-3ce7016ab6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.074669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d9a09-6495-4d93-9c11-3ce7016ab6f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.088345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d9a09-6495-4d93-9c11-3ce7016ab6f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:52.097674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8052db-0d2b-4bdd-b4e3-218307533d6a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3f1d9a09-6495-4d93-9c11-3ce7016ab6f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:52.098661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:52.099881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:52.100699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:52.103292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 280/562] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:52.104396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d9a09-6495-4d93-9c11-3ce7016ab6f5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.131184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:52.143403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.144173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.144451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.144722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.145695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:52.146111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.148545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.174107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 283/563] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 08:07:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:52.175048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.220611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d9a09-6495-4d93-9c11-3ce7016ab6f5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.256305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:52.258908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.260054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.260339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.260832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.267952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.269396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.282822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ada51-3b0b-43cc-b410-688d5ab34977 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:52.284407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:52.285293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:52.286219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 281/564] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:52.286799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.321809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4129e-2dda-4336-9144-b952f1de7435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.322537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4129e-2dda-4336-9144-b952f1de7435 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.322750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4129e-2dda-4336-9144-b952f1de7435 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.323020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4129e-2dda-4336-9144-b952f1de7435 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.323811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4129e-2dda-4336-9144-b952f1de7435 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:52.324116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4129e-2dda-4336-9144-b952f1de7435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.325524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4129e-2dda-4336-9144-b952f1de7435 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.338645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 284/565] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:52.340109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27681ddb-e7f3-460c-b803-8034b58eb64d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.340399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27681ddb-e7f3-460c-b803-8034b58eb64d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.340666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27681ddb-e7f3-460c-b803-8034b58eb64d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.341914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 285/566] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:52.342680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27681ddb-e7f3-460c-b803-8034b58eb64d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.374220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:52.394744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6f79f-0e39-414e-ba63-06d040836a00 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.429302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.430102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.430387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.430662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.441158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.442702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.453134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27681ddb-e7f3-460c-b803-8034b58eb64d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:52.461146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:52.462063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:52.462959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:52.462959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:52.462959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:52.462959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:52.462959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 282/567] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:52.463271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.496473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7c2a1-8039-49a4-9d4b-a43a4fd73069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.497531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7c2a1-8039-49a4-9d4b-a43a4fd73069 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.498107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7c2a1-8039-49a4-9d4b-a43a4fd73069 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.498401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7c2a1-8039-49a4-9d4b-a43a4fd73069 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.499390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7c2a1-8039-49a4-9d4b-a43a4fd73069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:52.499705 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7c2a1-8039-49a4-9d4b-a43a4fd73069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.501158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7c2a1-8039-49a4-9d4b-a43a4fd73069 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.512023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:52.522841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 286/568] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:52.524224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9ab4-53d5-40ce-99f2-023dcc00d57e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.524515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9ab4-53d5-40ce-99f2-023dcc00d57e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.524793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9ab4-53d5-40ce-99f2-023dcc00d57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.542526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bf3a9ab4-53d5-40ce-99f2-023dcc00d57e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:52.544151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bf3a9ab4-53d5-40ce-99f2-023dcc00d57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:52.567891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c3cbc-f2db-4977-95f7-69a2c0a16b91 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.598569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:52.605016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.605843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.606119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.606386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.617485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.619007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.623513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 287/569] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:52.625028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a9ab4-53d5-40ce-99f2-023dcc00d57e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:52.637143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:52.638247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:52.639161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:52.639161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:52.639161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:52.639161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:52.639161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 283/570] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:52.639161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.725756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a9ab4-53d5-40ce-99f2-023dcc00d57e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.741033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a2697-ea03-4dc5-b015-0a2ba4c7dd2a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.751685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:52.761703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37cc5d-2c3c-4475-97cf-a8631d1dc15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.762592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37cc5d-2c3c-4475-97cf-a8631d1dc15f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.762872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37cc5d-2c3c-4475-97cf-a8631d1dc15f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.763178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37cc5d-2c3c-4475-97cf-a8631d1dc15f None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.763929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37cc5d-2c3c-4475-97cf-a8631d1dc15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:52.764248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37cc5d-2c3c-4475-97cf-a8631d1dc15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.765645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37cc5d-2c3c-4475-97cf-a8631d1dc15f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.773823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.774470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.774750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.775022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.775871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=7356a0f8d9f84e7cb785662db9afd658) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:52.776180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.776762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 288/571] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:52.777595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.778005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082146-e874-4468-a767-267a576369e2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.778292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082146-e874-4468-a767-267a576369e2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.778569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082146-e874-4468-a767-267a576369e2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.779814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 289/572] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:52.780496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27082146-e874-4468-a767-267a576369e2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.786307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 284/573] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/role_assignments?user.id=7356a0f8d9f84e7cb785662db9afd658&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:52.787441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.845900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:52.902756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fccf25-ced6-49ad-9dc3-77c59f69899e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.904509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27082146-e874-4468-a767-267a576369e2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:52.941999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab78700-a36f-4575-a194-0dd9a8b0a83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.942380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:52.942840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab78700-a36f-4575-a194-0dd9a8b0a83e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.943118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab78700-a36f-4575-a194-0dd9a8b0a83e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.943346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.943503 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab78700-a36f-4575-a194-0dd9a8b0a83e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.943669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.943967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.944646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:52.944956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:52.946376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:52.954178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab78700-a36f-4575-a194-0dd9a8b0a83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:52.965397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 290/574] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:52.965397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2ab78700-a36f-4575-a194-0dd9a8b0a83e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:52.965649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:52.966689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 285/575] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:52.967998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63224434-9a6b-4f90-951e-a3607f5fd545 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:52.969118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63224434-9a6b-4f90-951e-a3607f5fd545 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:52.969118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63224434-9a6b-4f90-951e-a3607f5fd545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:52.983833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-63224434-9a6b-4f90-951e-a3607f5fd545 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:52.985023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-63224434-9a6b-4f90-951e-a3607f5fd545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:53.018347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:53.063372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 286/576] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:53.064545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63224434-9a6b-4f90-951e-a3607f5fd545 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.076377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cab33-371c-493b-8b36-34934d7dbd59 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.110848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:53.111729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.112733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.113160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.113592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.120677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.122473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:53.137258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:53.138162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:53.139035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:53.139035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:53.139035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:53.139035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:53.139035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:53.139035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:53.139035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:53.139035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:53.143305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 291/577] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:53.144064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.175323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63224434-9a6b-4f90-951e-a3607f5fd545 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.213323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.214132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.214417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.214695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.222445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.224031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:53.238974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:53.240335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:53.241308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 287/578] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:53.241887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.257886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b07e-7f1c-4d54-82f2-7ff061a0ed03 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.265697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:53.295271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.296211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.296646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.296917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.297661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:53.297966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.299469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.312203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 292/579] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:53.312203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.350599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:53.354547 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6819f7-f45a-4028-a29d-3a1494e26328 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.391240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabacd5-d0c8-478e-8e38-a9b6ab75fed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.392029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabacd5-d0c8-478e-8e38-a9b6ab75fed6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.392305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabacd5-d0c8-478e-8e38-a9b6ab75fed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.392572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabacd5-d0c8-478e-8e38-a9b6ab75fed6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.403356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabacd5-d0c8-478e-8e38-a9b6ab75fed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-eeabacd5-d0c8-478e-8e38-a9b6ab75fed6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:53.411328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:53.412597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:53.413439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 288/580] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:53.414128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabacd5-d0c8-478e-8e38-a9b6ab75fed6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.426040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa5212-ab0a-4000-98c6-fb422a8849ec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.462918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.463886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.464317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.464595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.465387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:53.465693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.467141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.480384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 293/581] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:53.481203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.514663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:53.526246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabacd5-d0c8-478e-8e38-a9b6ab75fed6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.561659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.562258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.562544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.562820 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.563692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:53.564005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.565441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.582966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 289/582] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:53.583947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.598681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0874c-97e7-4410-bc2d-b157a7d439f0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.603789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:53.635344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.636076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.636357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.636634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.647991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.649476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:53.670538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:53.671464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:53.672333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:53.672333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:53.672333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:53.672333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:53.672333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 294/583] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:53.672605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.696804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0309e-625d-42d9-85db-d181639a89fb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.734257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.734965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.735280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.735558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.736301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:53.736610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.738208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.751790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 290/584] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:53.751790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.756841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:53.776427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9e2fb-0c66-469c-91c6-2d9dfc68b160 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.813296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da17dc-57f3-41af-b122-39f7e5a6dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.814114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da17dc-57f3-41af-b122-39f7e5a6dc2d None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.814401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da17dc-57f3-41af-b122-39f7e5a6dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.814679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da17dc-57f3-41af-b122-39f7e5a6dc2d None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/d8cbcc9295074710aeacb35e4e1a4348/roles/fa8c35ad77b442fbbc0d20d458fe8731` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.831003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da17dc-57f3-41af-b122-39f7e5a6dc2d None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=d8cbcc9295074710aeacb35e4e1a4348, role_id=fa8c35ad77b442fbbc0d20d458fe8731)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.832526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da17dc-57f3-41af-b122-39f7e5a6dc2d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.835567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:53.847603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 295/585] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:53 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/d8cbcc9295074710aeacb35e4e1a4348/roles/fa8c35ad77b442fbbc0d20d458fe8731 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:53.856109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4af2c-baa6-48ab-b695-7cc058c97f19 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:53.856311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da17dc-57f3-41af-b122-39f7e5a6dc2d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:53.899109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:53.900133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:53.900454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:53.900794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:53.914357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:53.916045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:53.937938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:53.938919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:53.939834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:53.939834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:53.939834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:53.939834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:53.939834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 291/586] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:53.940919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.013184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da17dc-57f3-41af-b122-39f7e5a6dc2d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.046953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:54.054688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.055445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.055762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.056184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.056708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:54.057043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.058494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.072483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507251d-1e7a-4b33-9979-f76f2080d150 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.074433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 296/587] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:54.076132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.107559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:54.111321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.112789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.113092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.113512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.125735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.127479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:54.146101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:54.147248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:54.148375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:54.148375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:54.148375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:54.148375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:54.149768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 292/588] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:54.150484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.199640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193288c-e35b-4611-abe9-381354a298cc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.234951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.235959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.236436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.236848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.238176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:54.238176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.239886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.253419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 297/589] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:54.254367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.275830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f8f9f-8c66-45fc-b7f5-e9821f92ffe6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.308024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:54.320912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.322221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.322734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.323362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.334290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.335981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:54.354012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:54.355012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:07:54.356208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:54.358930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 293/590] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:54.360159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.382070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5692625-8cfe-4250-8d4c-c435b34aa090 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.407573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:54.423939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.424855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.425176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.425547 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.426381 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:54.426917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.428430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.440990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 298/591] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:54.442049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.481727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5018a-e692-4d10-8ab9-d038b23ef5fc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.525783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.527084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.528423 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.528856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.537837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.540094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.559848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd0a9-ee4f-4dd3-bf22-032ad06b4a3c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:54.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:54.561466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:54.562595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:54.562595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:54.562595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:54.562595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:54.562595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:54.562595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:54.562595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:54.562595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:54.565910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 294/592] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:54.566869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.574659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:54.603244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.604080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.604551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.604859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.605840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:54.606215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.607812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.630428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 299/593] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:54.631514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.698737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfc47-fb90-4fc9-be24-d081db019f4f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.701721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:54.747778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b72f85-fdba-410f-a3bc-b1f7a5df6dcd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.748396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.749612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.750043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.751351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.760684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.762671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:54.782628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:54.783704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 08:07:54.784824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:54.785412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 295/594] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:54.786300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.790433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.791144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.791471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.791743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.792938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:54.793051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.794520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.808247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 300/595] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:54.809112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.839189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:54.901344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c485bde-71df-46ba-8906-135b5301294d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.918513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d677541-8260-40dd-ad4f-98b468b7ac5d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:54.934872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.935733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.935986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.936264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.946959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.948451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.955004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178bb9b-5db4-4720-99c9-2d38289a5176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:54.955788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178bb9b-5db4-4720-99c9-2d38289a5176 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.956070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178bb9b-5db4-4720-99c9-2d38289a5176 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.956338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178bb9b-5db4-4720-99c9-2d38289a5176 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.957284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178bb9b-5db4-4720-99c9-2d38289a5176 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:54.957809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178bb9b-5db4-4720-99c9-2d38289a5176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:54.959540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178bb9b-5db4-4720-99c9-2d38289a5176 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:54.968942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:54.969887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:54.970822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:54.970822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:54.970822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:54.970822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:54.970822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 296/596] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:54.971103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.986638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 301/597] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:54.986638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0de5-7642-47b7-85e1-867889de9e41 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:54.986638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0de5-7642-47b7-85e1-867889de9e41 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:54.986638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0de5-7642-47b7-85e1-867889de9e41 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:54.986638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 302/598] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:54.986638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf0de5-7642-47b7-85e1-867889de9e41 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:54.997860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:55.078258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:55.084744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace02da8-6256-475f-b56e-2a6c9e84c25d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.095743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf0de5-7642-47b7-85e1-867889de9e41 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.122377 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44da0ef-5a10-4f85-9ac5-34594e360d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.123363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da0ef-5a10-4f85-9ac5-34594e360d7c None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.123658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da0ef-5a10-4f85-9ac5-34594e360d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.123904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da0ef-5a10-4f85-9ac5-34594e360d7c None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/419321fbd62c4d95a39cfd498b4a97d7/roles/fa8c35ad77b442fbbc0d20d458fe8731` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.133023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.133697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.133980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.134268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.134988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:55.135326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:55.136742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:55.139715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44da0ef-5a10-4f85-9ac5-34594e360d7c None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=419321fbd62c4d95a39cfd498b4a97d7, role_id=fa8c35ad77b442fbbc0d20d458fe8731)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:55.141176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44da0ef-5a10-4f85-9ac5-34594e360d7c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:55.148973 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 303/599] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:55.149809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.166699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 297/600] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:54 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/419321fbd62c4d95a39cfd498b4a97d7/roles/fa8c35ad77b442fbbc0d20d458fe8731 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:55.166699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdc10-69e1-4a92-b8c0-83f0221b8775 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.166699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdc10-69e1-4a92-b8c0-83f0221b8775 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.166699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdc10-69e1-4a92-b8c0-83f0221b8775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.182501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ba0cdc10-69e1-4a92-b8c0-83f0221b8775 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:55.183933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ba0cdc10-69e1-4a92-b8c0-83f0221b8775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:55.247411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:55.259537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 298/601] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:55.260555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cdc10-69e1-4a92-b8c0-83f0221b8775 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.282078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213245e-23f8-49a1-b313-2bf0449ce9a2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.327768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.328620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.328947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.329224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.344067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:55.345556 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:55.354318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:55.369454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:55.370478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:55.371599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:55.371599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:55.371599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:55.371599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:55.375397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 304/602] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:55.376284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.399397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cdc10-69e1-4a92-b8c0-83f0221b8775 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.444551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.446351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.446808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.447332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.456946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:55.459252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:55.477570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:55.478702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:55.479819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:55.479819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:55.479819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:55.479819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:55.479819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:55.479819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:55.479819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:55.479819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:55.481509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 299/603] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:55.482658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.500064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ca32e-4017-4cf5-bf75-ab768747dfb8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.542902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b333f-bb47-4a6a-93e5-39151060650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.544065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b333f-bb47-4a6a-93e5-39151060650a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.544489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b333f-bb47-4a6a-93e5-39151060650a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.544982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b333f-bb47-4a6a-93e5-39151060650a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.551885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:55.559384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b333f-bb47-4a6a-93e5-39151060650a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-804b333f-bb47-4a6a-93e5-39151060650a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:55.568212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:55.569222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:55.570048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:55.577579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 305/604] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:55.578832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b333f-bb47-4a6a-93e5-39151060650a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.602316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4a1f8-1e08-4896-bb54-106048327cf0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.646760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.647484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.648298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.648581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.649313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:55.649620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:55.651517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:55.655177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:55.673515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 300/605] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:55.673515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.707296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b333f-bb47-4a6a-93e5-39151060650a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.747880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.749354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.749899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.750373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.751434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:55.751941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:55.753667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:55.769723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 306/606] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:55.770771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.792524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafb2d0-88d9-4642-a62b-ad4423573b5d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.811780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:55.840096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.841083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.841505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.841915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.843028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:55.843497 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:55.845890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:55.856572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 301/607] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:55.857939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.882062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507acad7-5ec2-4d5d-ab2b-7c3511a98e29 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:55.900842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:55.919947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:55.920912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:55.921285 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:55.921672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:55.928939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:55.930520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:55.945162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:55.946261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 307/608] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:55.947177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:55.986042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0a299-64a2-4afe-9dd7-49cb05c4ec7c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:56.029802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212df896-e996-4e3b-91d1-fa0da2f49b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:56.030739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df896-e996-4e3b-91d1-fa0da2f49b73 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.030999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df896-e996-4e3b-91d1-fa0da2f49b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.031328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df896-e996-4e3b-91d1-fa0da2f49b73 None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/96456bfed82d469b888bd495ec46e796/roles/fa8c35ad77b442fbbc0d20d458fe8731` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.047297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212df896-e996-4e3b-91d1-fa0da2f49b73 None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=96456bfed82d469b888bd495ec46e796, role_id=fa8c35ad77b442fbbc0d20d458fe8731)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:56.052184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212df896-e996-4e3b-91d1-fa0da2f49b73 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:56.057904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02bb58-4469-415d-8c6d-8e1af44aa0e4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:56.070043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 302/609] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:55 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/96456bfed82d469b888bd495ec46e796/roles/fa8c35ad77b442fbbc0d20d458fe8731 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:56.071576 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86273dd-8902-4fb2-a23a-3197f205b034 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.071931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86273dd-8902-4fb2-a23a-3197f205b034 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.072273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86273dd-8902-4fb2-a23a-3197f205b034 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.077768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 303/610] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:56.077768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86273dd-8902-4fb2-a23a-3197f205b034 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:56.108433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b83f0-d052-4434-9200-ae4fdb21a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:56.109612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b83f0-d052-4434-9200-ae4fdb21a9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.109879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b83f0-d052-4434-9200-ae4fdb21a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.110155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b83f0-d052-4434-9200-ae4fdb21a9c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.117768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:56.123696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b83f0-d052-4434-9200-ae4fdb21a9c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:56.126735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b83f0-d052-4434-9200-ae4fdb21a9c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-024b83f0-d052-4434-9200-ae4fdb21a9c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:56.156299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:56.157425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:56.158529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:56.158529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:56.158529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:56.158529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:56.159826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 308/611] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:56.161278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695824e-1d68-4990-86b6-85fed9690e03 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.161927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695824e-1d68-4990-86b6-85fed9690e03 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.161927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695824e-1d68-4990-86b6-85fed9690e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.180712 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6695824e-1d68-4990-86b6-85fed9690e03 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:56.182014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6695824e-1d68-4990-86b6-85fed9690e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:56.212916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:56.223560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86273dd-8902-4fb2-a23a-3197f205b034 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:56.266847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:56.267937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.268499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.268804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.269605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:56.269915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:56.271650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:56.273605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 309/612] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:56.275328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695824e-1d68-4990-86b6-85fed9690e03 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:56.288776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 304/613] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:56.296175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:56.392407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695824e-1d68-4990-86b6-85fed9690e03 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:56.401796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:56.418505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce9a90-b311-4406-a5f6-46c0229c9b9f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:56.440678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:56.442232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.442936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.443410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.444283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:56.444975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:56.447415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:56.463082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 310/614] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:56.463896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64171d6-d8ef-4b33-92db-8fa87e237848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:56.464369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:56.464893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64171d6-d8ef-4b33-92db-8fa87e237848 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.465138 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64171d6-d8ef-4b33-92db-8fa87e237848 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.465417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64171d6-d8ef-4b33-92db-8fa87e237848 None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/2965b1c8ab52457f8f167acda4ba6844/roles/fa8c35ad77b442fbbc0d20d458fe8731` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.484030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64171d6-d8ef-4b33-92db-8fa87e237848 None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=2965b1c8ab52457f8f167acda4ba6844, role_id=fa8c35ad77b442fbbc0d20d458fe8731)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:56.485794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64171d6-d8ef-4b33-92db-8fa87e237848 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:56.498734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:56.511320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 305/615] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:56 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/2965b1c8ab52457f8f167acda4ba6844/roles/fa8c35ad77b442fbbc0d20d458fe8731 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:56.514205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64171d6-d8ef-4b33-92db-8fa87e237848 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:56.679271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b482e8-04bf-4c3a-8096-499803181309 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:56.705018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64171d6-d8ef-4b33-92db-8fa87e237848 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:56.735976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01ed2e-86c9-46db-a170-0bbc0aab3952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:56.737776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ed2e-86c9-46db-a170-0bbc0aab3952 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.738266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ed2e-86c9-46db-a170-0bbc0aab3952 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.739021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ed2e-86c9-46db-a170-0bbc0aab3952 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.750204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:56.751011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:56.751362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:56.751873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:56.752726 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01ed2e-86c9-46db-a170-0bbc0aab3952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:56.755315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:56.762544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:56.764726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bd01ed2e-86c9-46db-a170-0bbc0aab3952 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:56.769819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:56.771738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:56.772576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:56.774044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 311/616] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:56.778970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01ed2e-86c9-46db-a170-0bbc0aab3952 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:56.790673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:56.791666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:56.792586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 306/617] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:07:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:56.800439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:56.927632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:56.957246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01ed2e-86c9-46db-a170-0bbc0aab3952 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:56.979270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c0112-3216-4f23-a077-78d5d884a453 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:57.019039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:57.019750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.020030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.020330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.021090 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:57.021395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:57.022808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:57.038306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 312/618] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:57.040223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:57.040909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.041188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.041460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.042309 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:57.042617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:57.044075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:57.046561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:57.077810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 307/619] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 08:07:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:57.082366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:57.186320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:57.203221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28903a25-da02-44d4-a221-5a18877d9599 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:57.238313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00873082-693d-4abb-8f78-52892756c526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:57.239238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00873082-693d-4abb-8f78-52892756c526 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.239627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00873082-693d-4abb-8f78-52892756c526 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.239909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00873082-693d-4abb-8f78-52892756c526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.250860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00873082-693d-4abb-8f78-52892756c526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:57.252376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00873082-693d-4abb-8f78-52892756c526 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:57.259544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-00873082-693d-4abb-8f78-52892756c526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:57.273007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:57.274106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:57.274988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:57.274988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:57.274988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:57.274988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:57.274988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 313/620] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:57.279899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00873082-693d-4abb-8f78-52892756c526 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:57.302872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277af6eb-9474-4a56-b5f0-2c87b47cc4ef None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:57.374058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:57.374958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.375334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.375717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.397085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:57.398721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:57.431554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:57.432683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:57.433770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:57.433770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:57.433770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:57.433770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:57.435653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 308/621] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:57.440858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:57.446516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00873082-693d-4abb-8f78-52892756c526 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:57.484722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:57.488384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.488384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.488384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.488384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:57.488384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:57.499353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:57.525352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 314/622] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:57.536339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:57.579512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c9244-a95c-44b9-a506-c1d2fe81f9ff None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:57.587947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:57.616963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:57.619375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c340ad-c25f-4bd2-a147-d3a9aba91ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:57.620197 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c340ad-c25f-4bd2-a147-d3a9aba91ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.620793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c340ad-c25f-4bd2-a147-d3a9aba91ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.621244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c340ad-c25f-4bd2-a147-d3a9aba91ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.622445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c340ad-c25f-4bd2-a147-d3a9aba91ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:57.623296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c340ad-c25f-4bd2-a147-d3a9aba91ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:57.625689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c340ad-c25f-4bd2-a147-d3a9aba91ad2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:57.638434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 309/623] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:57.640006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be729bff-2f8d-4c77-8b33-1cd0851cdf49 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.640456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be729bff-2f8d-4c77-8b33-1cd0851cdf49 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.640965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be729bff-2f8d-4c77-8b33-1cd0851cdf49 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.642556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 310/624] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:57.643428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be729bff-2f8d-4c77-8b33-1cd0851cdf49 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:57.677214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59480873-58e9-4904-953d-fab514cb6ce3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:57.713188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce7ead-f2c4-4c65-934a-c39d5b0d17d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:57.714364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce7ead-f2c4-4c65-934a-c39d5b0d17d3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.714839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce7ead-f2c4-4c65-934a-c39d5b0d17d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.715336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce7ead-f2c4-4c65-934a-c39d5b0d17d3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.722987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce7ead-f2c4-4c65-934a-c39d5b0d17d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:57.724818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce7ead-f2c4-4c65-934a-c39d5b0d17d3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-91ce7ead-f2c4-4c65-934a-c39d5b0d17d3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:57.742422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:57.743419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 08:07:57.744382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:57.747306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 315/625] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:57.748721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d8bd3-c241-4c3c-8d07-7467a539a5f7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.749206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d8bd3-c241-4c3c-8d07-7467a539a5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.749678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d8bd3-c241-4c3c-8d07-7467a539a5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.767633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be729bff-2f8d-4c77-8b33-1cd0851cdf49 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:57.768154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-522d8bd3-c241-4c3c-8d07-7467a539a5f7 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:57.769932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-522d8bd3-c241-4c3c-8d07-7467a539a5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:57.812764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:57.813579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:57.813861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:57.814137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:57.823809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:57.825418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:57.843173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:57.844182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:57.845058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:57.845058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:57.845058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:57.845058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:57.845058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:57.845058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:57.845058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:57.845058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:57.846255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 316/626] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:57.847329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d8bd3-c241-4c3c-8d07-7467a539a5f7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:57.854273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 311/627] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:57.859965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:57.891899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:57.911771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:57.968408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d8bd3-c241-4c3c-8d07-7467a539a5f7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:57.991805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec1be8-e343-4de2-878e-cc53b124ce41 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.013524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.014710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.015634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.016235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.017386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:58.018136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:58.020442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:58.026445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.027373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.027554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.027807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.028591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:58.028947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:58.030360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:58.047986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 317/628] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 08:07:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:58.049099 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 312/629] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:07:58.049801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.049932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.157426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:58.170368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48da34-49ab-4bc8-8936-b92a61a71db5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.172190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:58.179879 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9ce8b-a212-4d4b-85a3-8655ad66cbbc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.213834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011d74e-83dc-4b0c-a70e-bce8863be0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.214642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011d74e-83dc-4b0c-a70e-bce8863be0ea None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.214919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011d74e-83dc-4b0c-a70e-bce8863be0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.215190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011d74e-83dc-4b0c-a70e-bce8863be0ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.224236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f4f07-c5f9-47c2-9e86-fdc60e609447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.225150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f4f07-c5f9-47c2-9e86-fdc60e609447 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.225411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f4f07-c5f9-47c2-9e86-fdc60e609447 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.225683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f4f07-c5f9-47c2-9e86-fdc60e609447 None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/7356a0f8d9f84e7cb785662db9afd658/roles/fa8c35ad77b442fbbc0d20d458fe8731` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.225856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011d74e-83dc-4b0c-a70e-bce8863be0ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a011d74e-83dc-4b0c-a70e-bce8863be0ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:58.233545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:58.234470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 318/630] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:58.236102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011d74e-83dc-4b0c-a70e-bce8863be0ea None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.246342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f4f07-c5f9-47c2-9e86-fdc60e609447 None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=7356a0f8d9f84e7cb785662db9afd658, role_id=fa8c35ad77b442fbbc0d20d458fe8731)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:58.248472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f4f07-c5f9-47c2-9e86-fdc60e609447 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:58.263160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 313/631] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:07:58 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/7356a0f8d9f84e7cb785662db9afd658/roles/fa8c35ad77b442fbbc0d20d458fe8731 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:07:58.271898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbfacd-b2be-43ff-8fda-0feb52b66218 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.272082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbfacd-b2be-43ff-8fda-0feb52b66218 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.272356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbfacd-b2be-43ff-8fda-0feb52b66218 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.275249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 314/632] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:58.275249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbfacd-b2be-43ff-8fda-0feb52b66218 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.366242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011d74e-83dc-4b0c-a70e-bce8863be0ea None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.405996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbfacd-b2be-43ff-8fda-0feb52b66218 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.411629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233818d1-c6e8-47b4-8b06-73f1b301fa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.412541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233818d1-c6e8-47b4-8b06-73f1b301fa60 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.412993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233818d1-c6e8-47b4-8b06-73f1b301fa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.413420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233818d1-c6e8-47b4-8b06-73f1b301fa60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.451442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45647e-b2b7-470f-8f94-89e4584fe1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.452058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:58.453523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45647e-b2b7-470f-8f94-89e4584fe1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.453655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45647e-b2b7-470f-8f94-89e4584fe1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.453866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45647e-b2b7-470f-8f94-89e4584fe1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.454652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45647e-b2b7-470f-8f94-89e4584fe1e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:58.455013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45647e-b2b7-470f-8f94-89e4584fe1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:58.456724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45647e-b2b7-470f-8f94-89e4584fe1e2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:58.460031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:58.468769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 315/633] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:58.470708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa520ee-c049-4487-9154-aa6f1a89692a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.470994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa520ee-c049-4487-9154-aa6f1a89692a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.471885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa520ee-c049-4487-9154-aa6f1a89692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.488708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4aa520ee-c049-4487-9154-aa6f1a89692a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:58.490107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4aa520ee-c049-4487-9154-aa6f1a89692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:58.520682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233818d1-c6e8-47b4-8b06-73f1b301fa60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:58.523353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233818d1-c6e8-47b4-8b06-73f1b301fa60 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:58.565970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 316/634] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:58.566832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa520ee-c049-4487-9154-aa6f1a89692a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.646192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 319/635] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:07:58.647647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233818d1-c6e8-47b4-8b06-73f1b301fa60 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.694165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa520ee-c049-4487-9154-aa6f1a89692a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.721213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:58.729280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.730527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.731008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.731581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.742759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:58.744379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:58.744524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:58.757790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233818d1-c6e8-47b4-8b06-73f1b301fa60 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:58.762828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:58.763772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:58.764690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:58.764690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:58.764690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:58.764690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:58.764690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 317/636] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:58.765244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.804400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a443c2-ad26-436f-bc37-a5862a7ca973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.805963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a443c2-ad26-436f-bc37-a5862a7ca973 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.806313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a443c2-ad26-436f-bc37-a5862a7ca973 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.806666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a443c2-ad26-436f-bc37-a5862a7ca973 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.819105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a443c2-ad26-436f-bc37-a5862a7ca973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-84a443c2-ad26-436f-bc37-a5862a7ca973 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:58.830144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:58.831163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:58.832067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:58.832864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 320/637] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:58.834208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e03fe-d82e-49c7-bae3-c26b656404c1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.834619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e03fe-d82e-49c7-bae3-c26b656404c1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.835019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e03fe-d82e-49c7-bae3-c26b656404c1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.839246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 321/638] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:07:58.839246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e03fe-d82e-49c7-bae3-c26b656404c1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.886921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f9947-d372-4f6c-9e7d-e5a861626a85 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.922935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:58.923769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:58.924051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:58.924324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:58.936172 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:58.937634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:58.955799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e03fe-d82e-49c7-bae3-c26b656404c1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:58.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:58.958643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:58.960752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:58.960752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:58.960752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:58.960752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:58.960752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 318/639] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:58.960752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:58.986975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:58.993573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:59.002009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450479c4-38b6-4a5c-916b-1899a50042d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.002680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450479c4-38b6-4a5c-916b-1899a50042d8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.003411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450479c4-38b6-4a5c-916b-1899a50042d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.003693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450479c4-38b6-4a5c-916b-1899a50042d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.004420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450479c4-38b6-4a5c-916b-1899a50042d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:59.005116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450479c4-38b6-4a5c-916b-1899a50042d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.006849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450479c4-38b6-4a5c-916b-1899a50042d8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.019588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 322/640] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:59.020917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679386d8-ee1d-4f73-b460-c3ce25a43ae3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.021198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679386d8-ee1d-4f73-b460-c3ce25a43ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.021468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679386d8-ee1d-4f73-b460-c3ce25a43ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.037129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-679386d8-ee1d-4f73-b460-c3ce25a43ae3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:07:59.038292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-679386d8-ee1d-4f73-b460-c3ce25a43ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:07:59.073316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d1869-87c2-412b-9e43-fba61332671b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.111236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 323/641] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:07:59.111236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679386d8-ee1d-4f73-b460-c3ce25a43ae3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.126499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.127302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.127732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.128079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.129020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:59.129403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.131507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.141073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 319/642] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:59.142231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.209112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679386d8-ee1d-4f73-b460-c3ce25a43ae3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.222845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:59.246068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.247064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.247380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.247660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.248808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:59.249124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.250635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.254910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:59.267236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 324/643] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:59.267439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd788b-928a-4620-84c6-bb50021003e9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.267544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.303482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.304692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.304692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.304896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.312700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.314160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:07:59.329219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:59.330293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 320/644] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:59.331895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.380413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821acc0e-dca9-4c2e-b271-39665de9ef98 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.415899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce64caf-f7de-4c29-9d0e-1f81b2cd2f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.417468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce64caf-f7de-4c29-9d0e-1f81b2cd2f03 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.417819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce64caf-f7de-4c29-9d0e-1f81b2cd2f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.418164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce64caf-f7de-4c29-9d0e-1f81b2cd2f03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.430525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce64caf-f7de-4c29-9d0e-1f81b2cd2f03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1ce64caf-f7de-4c29-9d0e-1f81b2cd2f03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:07:59.438485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:59.439645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:07:59.440737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:59.444763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 325/645] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:59.445638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce64caf-f7de-4c29-9d0e-1f81b2cd2f03 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.445882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa81498-3f9e-457a-b17c-7f5c8afa0c72 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.481021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:59.484051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.485076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.485363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.485694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.499183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.500703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:07:59.518786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:59.519701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:59.520788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:07:59.520788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:07:59.520788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:07:59.520788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:07:59.520788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 321/646] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:59.521734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.532665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:59.566284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce64caf-f7de-4c29-9d0e-1f81b2cd2f03 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.601442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.602283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.602637 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.603003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.612830 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.614636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:07:59.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:07:59.630246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:07:59.631336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:07:59.631336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:07:59.631336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:07:59.631336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:07:59.631336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:07:59.631336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:07:59.631336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:07:59.631336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:07:59.635010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 326/647] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:07:59.635865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.637474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d0cb0-1f4f-4729-8594-f190d8cfd620 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.671148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.671858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.672143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.672412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.673247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:59.673557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.674990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.698119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 322/648] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:59.698119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.738282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:59.748908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db1161-8a6d-4af3-8b07-7eff31b15024 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.760227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:59.786523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.787123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.787434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.787705 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.788480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:59.788788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.790176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.798444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 327/649] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:59.800075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.803395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78c80-9e06-44ec-bb67-666e6a527731 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.839752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.840714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.841179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.841654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.842580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:59.843077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.844676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.857043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 323/650] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:59.857862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.912373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca10d1-b2cf-4fe7-9a05-51a16439d854 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.947784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:07:59.948457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:07:59.948740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:07:59.949018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:07:59.949805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:07:59.950112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:07:59.951585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:07:59.965079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 328/651] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:07:59.965725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:07:59.970766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc2e51-a15c-4e74-96a6-547d80090d06 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:07:59.982860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:07:59.998981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:00.011358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a31d-b9d2-422c-b0a5-15fc44c98a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.012022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a31d-b9d2-422c-b0a5-15fc44c98a3c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.012306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a31d-b9d2-422c-b0a5-15fc44c98a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.012584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a31d-b9d2-422c-b0a5-15fc44c98a3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.076571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e931e27-3f32-42f0-a3de-a69156f26397 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.084710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324a31d-b9d2-422c-b0a5-15fc44c98a3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.086184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324a31d-b9d2-422c-b0a5-15fc44c98a3c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.114143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.114832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.115129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.115438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.125924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.127431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:00.145354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:00.146299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:00.147177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:00.147177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:00.147177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:00.147177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:00.147177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 329/652] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:00.147477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.197307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 324/653] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:08:00.198615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a31d-b9d2-422c-b0a5-15fc44c98a3c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.222929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:00.227734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:00.264090 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2e7bb-ebee-4f8f-a501-e8cd5109f60d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.305180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.306097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.306210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a31d-b9d2-422c-b0a5-15fc44c98a3c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.306392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.306646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.314723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.316226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:00.330927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:00.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 330/654] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:00.332980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.340136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.340794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.341077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.341349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.342170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=d8cbcc9295074710aeacb35e4e1a4348) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:00.342481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.343944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.349878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 325/655] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/role_assignments?user.id=d8cbcc9295074710aeacb35e4e1a4348&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:00.350664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.440015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50363974-e492-4a7b-a2b9-334e3ce0ea47 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.451447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:00.467945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd219838-9e30-49dc-9639-a712b6fa1254 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.470142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:00.482515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a153d8d-053f-43dc-a630-3a4b969ae755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.483374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a153d8d-053f-43dc-a630-3a4b969ae755 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.486651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a153d8d-053f-43dc-a630-3a4b969ae755 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.486651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a153d8d-053f-43dc-a630-3a4b969ae755 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.486651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a153d8d-053f-43dc-a630-3a4b969ae755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:00.486651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a153d8d-053f-43dc-a630-3a4b969ae755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.486994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a153d8d-053f-43dc-a630-3a4b969ae755 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.499815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 331/656] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:00.502709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21acb1b-29c1-484d-b6b8-84207a7234a2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.502709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21acb1b-29c1-484d-b6b8-84207a7234a2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.502709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21acb1b-29c1-484d-b6b8-84207a7234a2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.503096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 332/657] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:00.503719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21acb1b-29c1-484d-b6b8-84207a7234a2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.520020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.521016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.521306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.521586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.533060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.534518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:00.553617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:00.554560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:00.555870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:00.555870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:00.555870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:00.555870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:00.555870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 326/658] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:00.555870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.627012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21acb1b-29c1-484d-b6b8-84207a7234a2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.662490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed46c9-9177-4c43-b9bc-1dca9deaf834 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.665504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e081f-c314-466f-bc79-b326cc98b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.665504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e081f-c314-466f-bc79-b326cc98b1ab None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.665814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e081f-c314-466f-bc79-b326cc98b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.666032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e081f-c314-466f-bc79-b326cc98b1ab None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.666866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e081f-c314-466f-bc79-b326cc98b1ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:00.667175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e081f-c314-466f-bc79-b326cc98b1ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.668650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e081f-c314-466f-bc79-b326cc98b1ab None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.693089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 333/659] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:00.693089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbebdf-771f-4cbe-8f68-4da171932957 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.693089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbebdf-771f-4cbe-8f68-4da171932957 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.693089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbebdf-771f-4cbe-8f68-4da171932957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.698361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.699181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.699488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.699756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.707363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.708948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.709622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ffcbebdf-771f-4cbe-8f68-4da171932957 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:00.710804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ffcbebdf-771f-4cbe-8f68-4da171932957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:00.712053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:00.725215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:00.726221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:00.727143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 327/660] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:00.728800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.759765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:00.799357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 334/661] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:08:00.799357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbebdf-771f-4cbe-8f68-4da171932957 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.838008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dffc0-c6a3-4d5a-9bdb-944b755279aa None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.874307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.875078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.875380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.875649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.876375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:00.876682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:00.878057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:00.895727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 328/662] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:00.895727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:00.919915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbebdf-771f-4cbe-8f68-4da171932957 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:00.953584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:00.957096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d2e1-6c42-4db6-b144-dc6b34720640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:00.957817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d2e1-6c42-4db6-b144-dc6b34720640 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:00.958093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d2e1-6c42-4db6-b144-dc6b34720640 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:00.958367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d2e1-6c42-4db6-b144-dc6b34720640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:00.997206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57bcd-d6a3-49d2-8d7c-394b120eb03f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.001572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:01.027899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7d2e1-6c42-4db6-b144-dc6b34720640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.029346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7d2e1-6c42-4db6-b144-dc6b34720640 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.032985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc98c-f633-408c-8249-98a83674781b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.033675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc98c-f633-408c-8249-98a83674781b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.033960 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc98c-f633-408c-8249-98a83674781b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.034230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc98c-f633-408c-8249-98a83674781b None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.035042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abc98c-f633-408c-8249-98a83674781b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:01.035395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abc98c-f633-408c-8249-98a83674781b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.036840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abc98c-f633-408c-8249-98a83674781b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.056549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 329/663] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:01.057382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc98c-f633-408c-8249-98a83674781b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.151471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 335/664] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:08:01.151471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d2e1-6c42-4db6-b144-dc6b34720640 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.183223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc98c-f633-408c-8249-98a83674781b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.205122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:01.225107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a06ca6-da0d-440b-b389-456ccf321017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.226149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06ca6-da0d-440b-b389-456ccf321017 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.226589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06ca6-da0d-440b-b389-456ccf321017 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.226997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06ca6-da0d-440b-b389-456ccf321017 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.230673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:01.238435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a06ca6-da0d-440b-b389-456ccf321017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e7a06ca6-da0d-440b-b389-456ccf321017 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:01.246216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:01.247544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:01.248397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 330/665] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:01.249862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a06ca6-da0d-440b-b389-456ccf321017 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.261683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d2e1-6c42-4db6-b144-dc6b34720640 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.297424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.298219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.298500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.298773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.305639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.307104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:01.321020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:01.321894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:01.322781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 336/666] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:01.324662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.357116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a06ca6-da0d-440b-b389-456ccf321017 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.392888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682774c9-70be-4d20-b907-52b3958d088c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.393784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682774c9-70be-4d20-b907-52b3958d088c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.394066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682774c9-70be-4d20-b907-52b3958d088c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.394326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682774c9-70be-4d20-b907-52b3958d088c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.427104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e31c1e-1820-4085-9cb4-8ba107d872d6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.448488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:01.464692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.465802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.466274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.466737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.470459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:01.474762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682774c9-70be-4d20-b907-52b3958d088c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.476620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682774c9-70be-4d20-b907-52b3958d088c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.489576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.491299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:01.513812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:01.514927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:01.516033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:01.516033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:01.516033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:01.516033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:01.517214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 337/667] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:01.517587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.593667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 331/668] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:08:01.593667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682774c9-70be-4d20-b907-52b3958d088c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.631263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653f584-bf44-400a-81c2-95e571b046af None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.665466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.666274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.666619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.666959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.667823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:01.668194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.669656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.671637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:01.682210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 338/669] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:01.682210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.699155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682774c9-70be-4d20-b907-52b3958d088c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.730452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:01.735984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.736723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.737078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.737426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.738289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:01.738663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.740326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.757318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 332/670] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:01.757318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.789269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa5ea6-7f3a-499c-b5df-7f9cbc4993b1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.825926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.826877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.827333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.827756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.828620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:01.829064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.830601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.843364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 339/671] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:01.844738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.856091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52f3a-7327-42e1-8c23-1245e652173a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.890917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:01.891754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:01.892038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:01.892312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:01.900320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:01.906812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:01.908291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:01.925306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:01.926193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:01.927650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:01.927650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:01.927650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:01.927650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:01.931060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 333/672] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:01.931765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:01.959823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1709ff2-cef6-4c1b-85a9-0fc7d5a31ad7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:01.971625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:02.006236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.007317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.007746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.008264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.019495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.021123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.036591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5adab2-7674-4808-887c-cd9f0bffaf93 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:02.038795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:02.040566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:02.041455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:02.041455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:02.041455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:02.041455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:02.041455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 340/673] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:02.041455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.076179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.077274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.077736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.078007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.078833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=419321fbd62c4d95a39cfd498b4a97d7) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:02.079143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.080943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.095247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 334/674] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:01 2022] GET /identity/v3/role_assignments?user.id=419321fbd62c4d95a39cfd498b4a97d7&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:02.095247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.153309 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:02.191737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad782fdc-91b6-4203-b6c7-708df3e0d093 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.210273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df981831-d1a8-4cff-a1f1-c605755e56eb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.242375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.244248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.244890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.245467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.252687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.253366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.253647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.253927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.254732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.254858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:02.255178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.256362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.256734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:02.275866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:02.276836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:02.277735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:02.277735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:02.277735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:02.277735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:02.277735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:02.277735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:02.277735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:02.277735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:02.279093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 335/676] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:02.279290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 341/676] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:02.280142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.280985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.309345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:02.395229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e587f-b7bc-4c58-90b8-8dfd40d55539 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.399842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d60816-1c11-447f-8838-0bf59f61912a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.410591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:02.432576 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6be9fd-8415-4892-989f-4d14295dedd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.433481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6be9fd-8415-4892-989f-4d14295dedd1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.433776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6be9fd-8415-4892-989f-4d14295dedd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.434053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6be9fd-8415-4892-989f-4d14295dedd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.439618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.440563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.441038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.441502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.441810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6be9fd-8415-4892-989f-4d14295dedd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.442969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:02.443519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.443648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6be9fd-8415-4892-989f-4d14295dedd1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.444895 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.457204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 342/677] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9a6be9fd-8415-4892-989f-4d14295dedd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:02.458532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:02.459682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:02.460781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:02.460781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:02.460781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:02.460781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:02.460781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:02.460781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:02.460781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:02.460781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:02.461208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.461279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 336/678] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:02.461279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f656690-1e67-447d-94bd-9949c04d0b46 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.461279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f656690-1e67-447d-94bd-9949c04d0b46 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.461279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f656690-1e67-447d-94bd-9949c04d0b46 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.463348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 337/679] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:02.467951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f656690-1e67-447d-94bd-9949c04d0b46 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.549922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:02.570723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f38b5-eb77-4a04-b789-c0fc58ff4646 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.582905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f656690-1e67-447d-94bd-9949c04d0b46 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.608228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.609304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.609748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.610014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.610824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:02.611226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.612968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.618127 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242c690-bfcd-4f42-844b-c72748dd0c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.618821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242c690-bfcd-4f42-844b-c72748dd0c70 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.619111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242c690-bfcd-4f42-844b-c72748dd0c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.619407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242c690-bfcd-4f42-844b-c72748dd0c70 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.620187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242c690-bfcd-4f42-844b-c72748dd0c70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:02.620492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242c690-bfcd-4f42-844b-c72748dd0c70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.622241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242c690-bfcd-4f42-844b-c72748dd0c70 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.631771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 343/680] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:02.631771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.637964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 338/681] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:02.639193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6a947-dc74-44eb-973f-2ce84bfc35fe None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.639510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6a947-dc74-44eb-973f-2ce84bfc35fe None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.639750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6a947-dc74-44eb-973f-2ce84bfc35fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.658574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0ec6a947-dc74-44eb-973f-2ce84bfc35fe None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:02.659797 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0ec6a947-dc74-44eb-973f-2ce84bfc35fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:02.686977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:02.729742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 339/682] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:08:02.730857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6a947-dc74-44eb-973f-2ce84bfc35fe None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.745268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d92a8-66ba-400d-a0ca-50fa32d6d1fd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.780301 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.781408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.781865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.782322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.790783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.792516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.795305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:02.807852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:02.808902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:08:02.809858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:02.813819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 344/683] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:02.814573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.847191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6a947-dc74-44eb-973f-2ce84bfc35fe None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.884313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.884979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.885265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.885542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.886558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=96456bfed82d469b888bd495ec46e796) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:02.886737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.888200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.903260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 340/684] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/role_assignments?user.id=96456bfed82d469b888bd495ec46e796&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:02.903260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:02.933272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e905d-e2ef-40eb-b8ed-ec1830832d59 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:02.940622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:02.978749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:02.979644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:02.979990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:02.980433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:02.981337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=2965b1c8ab52457f8f167acda4ba6844) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:02.981709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:02.983420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:02.995787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 345/685] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/role_assignments?user.id=2965b1c8ab52457f8f167acda4ba6844&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:02.997313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:03.039296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e8083-4940-43a5-bd54-16011b198781 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:03.124801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:03.125627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:03.125904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:03.126173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:03.127527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:03.140128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:03.141605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:03.161116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:03.162128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:03.163020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:03.163020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:03.163020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:03.163020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:03.167451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 341/686] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:03.168643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:03.176818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83349cb6-53de-452f-a1b7-33b80d3af3d1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:03.223800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:03.224585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:03.224863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:03.225157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:03.225963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:03.226268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:03.227944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:03.250341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 346/687] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 08:08:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:03.256133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:03.274336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:03.306226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5f61-dfd5-4e7a-abec-2d9412a6b7c4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:03.342429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:03.343098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:03.343476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:03.343750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:03.344476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:03.344779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:03.346160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:03.359731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 342/688] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:03.372528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:03.372729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c105b6c-0e40-4618-95bc-c859dc19ecd1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:03.399326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:03.411899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8269a9-e5d1-48eb-baad-7906c0c2890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:03.412563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8269a9-e5d1-48eb-baad-7906c0c2890a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:03.412846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8269a9-e5d1-48eb-baad-7906c0c2890a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:03.413179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8269a9-e5d1-48eb-baad-7906c0c2890a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:03.481385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0219ba-d942-4340-9888-fe690bd8c10e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:03.486884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8269a9-e5d1-48eb-baad-7906c0c2890a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:03.488551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8269a9-e5d1-48eb-baad-7906c0c2890a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:03.545654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47a79a-48f8-4a05-b79e-c16d13cf1455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:03.546622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47a79a-48f8-4a05-b79e-c16d13cf1455 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:03.547048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47a79a-48f8-4a05-b79e-c16d13cf1455 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:03.547492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47a79a-48f8-4a05-b79e-c16d13cf1455 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:03.551058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47a79a-48f8-4a05-b79e-c16d13cf1455 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:03.552972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47a79a-48f8-4a05-b79e-c16d13cf1455 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:03.573444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:03.599667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 343/689] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:08:03 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:03.620782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 347/690] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 08:08:03.627159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47a79a-48f8-4a05-b79e-c16d13cf1455 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:03.774266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47a79a-48f8-4a05-b79e-c16d13cf1455 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:03.825428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:03.826163 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:03.826442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:03.826713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:03.847679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:03.849204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:03.870600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:03.871597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:03.872827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:03.872827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:03.872827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:03.872827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:03.872827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 344/691] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:03.877833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8269a9-e5d1-48eb-baad-7906c0c2890a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:03.913675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:04.010627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8269a9-e5d1-48eb-baad-7906c0c2890a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:04.047612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:04.048676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:04.049147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:04.049638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:04.050558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:04.051075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:04.053043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:04.066204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 348/692] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:04.071657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:04.152528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:04.216060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8230fe-9c27-49bb-93ca-d63f318e7385 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:04.259136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:04.259976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:04.260258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:04.260528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:04.267622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:04.269072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:04.283739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:04.284709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:04.285592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 345/693] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:04.291139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:04.442362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:04.470999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c785c-ddce-4e2a-b00c-537c2f0c9aaf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:04.536536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944071b-106b-4062-9d37-0ea6bb4dba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:04.537517 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944071b-106b-4062-9d37-0ea6bb4dba92 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:04.538001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944071b-106b-4062-9d37-0ea6bb4dba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:04.538460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944071b-106b-4062-9d37-0ea6bb4dba92 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:04.539441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944071b-106b-4062-9d37-0ea6bb4dba92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:04.539924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944071b-106b-4062-9d37-0ea6bb4dba92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:04.541612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944071b-106b-4062-9d37-0ea6bb4dba92 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:04.558804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 349/694] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:04.564519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:04.583080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fe8af-7d5f-48ba-8e58-18e9375b193f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:04.583718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fe8af-7d5f-48ba-8e58-18e9375b193f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:04.584169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fe8af-7d5f-48ba-8e58-18e9375b193f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:04.585711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 350/695] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:04.590696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd881f8-3473-49c9-bac1-62845a0c99a7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:04.591165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd881f8-3473-49c9-bac1-62845a0c99a7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:04.591657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd881f8-3473-49c9-bac1-62845a0c99a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:04.630657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4fd881f8-3473-49c9-bac1-62845a0c99a7 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:04.631870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4fd881f8-3473-49c9-bac1-62845a0c99a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:04.722886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3d48a-b5ee-4e36-90e9-426a5ebceb7a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:04.739536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:04.768360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 351/696] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:08:04.779139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:04.779850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:04.780135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:04.780413 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:04.781252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=7356a0f8d9f84e7cb785662db9afd658) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:04.781567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:04.783018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:04.789317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 346/697] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:04 2022] GET /identity/v3/role_assignments?user.id=7356a0f8d9f84e7cb785662db9afd658&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:04.802915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd881f8-3473-49c9-bac1-62845a0c99a7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:04.875694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:04.922401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd881f8-3473-49c9-bac1-62845a0c99a7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:04.974586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:04.975948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:04.976427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:04.976901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:04.977773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:04.979414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:05.006884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:05.008200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:05.009038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:05.011998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 352/698] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 23 08:08:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:05.018452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:05.154821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482c20-45e5-4fa5-8277-adb5343ee397 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:05.199554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed3c34-173a-4be8-9118-089c4b1b41b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:05.200620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed3c34-173a-4be8-9118-089c4b1b41b5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:05.201106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed3c34-173a-4be8-9118-089c4b1b41b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:05.201570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed3c34-173a-4be8-9118-089c4b1b41b5 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:05.202470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed3c34-173a-4be8-9118-089c4b1b41b5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:05.203029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed3c34-173a-4be8-9118-089c4b1b41b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:05.204661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed3c34-173a-4be8-9118-089c4b1b41b5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:05.210069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 347/699] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 08:08:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:05.215243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:05.359530 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68fc63-e179-45f9-8cc0-a2e798ca8262 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:05.406288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4d558-a1c4-43cd-9a20-72a0c53233d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:05.406968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4d558-a1c4-43cd-9a20-72a0c53233d4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:05.407276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4d558-a1c4-43cd-9a20-72a0c53233d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:05.407555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4d558-a1c4-43cd-9a20-72a0c53233d4 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:05.408273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4d558-a1c4-43cd-9a20-72a0c53233d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:05.408580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4d558-a1c4-43cd-9a20-72a0c53233d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:05.409985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4d558-a1c4-43cd-9a20-72a0c53233d4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:05.417151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:05.420646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 353/700] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 08:08:05 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:05.506450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86039f43-a815-4c2b-9b38-31e03ea48e21 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:05.506450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86039f43-a815-4c2b-9b38-31e03ea48e21 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:05.506742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86039f43-a815-4c2b-9b38-31e03ea48e21 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:05.508282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 348/701] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:05.513704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429829fd-2e4e-44e2-aefe-29fc41f6dc88 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:05.514006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429829fd-2e4e-44e2-aefe-29fc41f6dc88 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:05.514289 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429829fd-2e4e-44e2-aefe-29fc41f6dc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:05.531852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-429829fd-2e4e-44e2-aefe-29fc41f6dc88 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:05.533047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-429829fd-2e4e-44e2-aefe-29fc41f6dc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:05.588836 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421a9f6-51fb-4b1e-aba5-efd064fcb07b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:05.589449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421a9f6-51fb-4b1e-aba5-efd064fcb07b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:05.589927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421a9f6-51fb-4b1e-aba5-efd064fcb07b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:05.591811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 349/702] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:05.597455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a47c-2fe7-4d4b-98d3-1d2809e29fe3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:05.598023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a47c-2fe7-4d4b-98d3-1d2809e29fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:05.598524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a47c-2fe7-4d4b-98d3-1d2809e29fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:05.629081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 354/703] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:08:05.630454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2c22a47c-2fe7-4d4b-98d3-1d2809e29fe3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:05.631967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2c22a47c-2fe7-4d4b-98d3-1d2809e29fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:05.663894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:05.670873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429829fd-2e4e-44e2-aefe-29fc41f6dc88 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:05.722775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 350/704] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:08:05.783246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22a47c-2fe7-4d4b-98d3-1d2809e29fe3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:05.809553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429829fd-2e4e-44e2-aefe-29fc41f6dc88 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:05.839696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:05.850345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:05.851174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:05.851479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:05.851753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] PATH_INFO: `/v3/services/network` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:05.852421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:05.853835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:05.879744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:05.880790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:05.881633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:05.888619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 355/705] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:05 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:05.889501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:05.988119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22a47c-2fe7-4d4b-98d3-1d2809e29fe3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:06.028584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686deaa0-aa7b-4f60-822f-187201fe27df None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:06.056399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ad192-e399-4649-89a8-20c172b924ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:06.057909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ad192-e399-4649-89a8-20c172b924ad None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.058577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ad192-e399-4649-89a8-20c172b924ad None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.059268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ad192-e399-4649-89a8-20c172b924ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.073018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ad192-e399-4649-89a8-20c172b924ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:06.077246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:06.077930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.078216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.078494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.079231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] RBAC: Adding query filter params (name=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:06.079548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:06.080954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-700ad192-e399-4649-89a8-20c172b924ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:06.083105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:06.084056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:06.084903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:06.087675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 351/706] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:06.093293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ad192-e399-4649-89a8-20c172b924ad None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:06.097656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 356/707] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:06.102356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:06.151910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:06.220045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:06.271635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf684c-d0d4-43d3-8867-c743427d259a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:06.289557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ad192-e399-4649-89a8-20c172b924ad None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:06.322305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5071e-71ed-4a79-8c6c-9d178861964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:06.323227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5071e-71ed-4a79-8c6c-9d178861964d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.323780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5071e-71ed-4a79-8c6c-9d178861964d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.324162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5071e-71ed-4a79-8c6c-9d178861964d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.325008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5071e-71ed-4a79-8c6c-9d178861964d None admin] RBAC: Adding query filter params (type=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:06.325442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5071e-71ed-4a79-8c6c-9d178861964d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:06.327433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5071e-71ed-4a79-8c6c-9d178861964d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:06.333472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 357/708] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:06.334806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a88024-c9cf-4019-887f-2fdf3dabf592 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.335095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a88024-c9cf-4019-887f-2fdf3dabf592 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.335396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a88024-c9cf-4019-887f-2fdf3dabf592 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.336976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 358/709] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:06.342186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371814-50ec-4c83-ad62-a4384df89049 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.342516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371814-50ec-4c83-ad62-a4384df89049 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.342747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371814-50ec-4c83-ad62-a4384df89049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.352113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:06.352935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.353223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.353503 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.354277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:06.354591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:06.356041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:06.365130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1c371814-50ec-4c83-ad62-a4384df89049 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:06.366762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1c371814-50ec-4c83-ad62-a4384df89049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:06.371392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 352/710] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:06.377284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:06.484862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 359/711] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 08:08:06.500573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:06.512405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c371814-50ec-4c83-ad62-a4384df89049 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:06.534896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2befe-3477-48e4-b34b-1d7bbecf7f6f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:06.599280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:06.601703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:06.602577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.602851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.603118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.622689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:06.624231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:06.646395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:06.647425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:06.648328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:06.648328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:06.648328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:06.648328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:06.648328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 353/712] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:06.654070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:06.691022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c371814-50ec-4c83-ad62-a4384df89049 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:06.761879 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1e77-dc94-41e2-96c4-d405587cc007 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:06.784071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e67d6-771f-460c-ba11-5c01a6b01503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:06.785200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e67d6-771f-460c-ba11-5c01a6b01503 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.785785 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e67d6-771f-460c-ba11-5c01a6b01503 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.786208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e67d6-771f-460c-ba11-5c01a6b01503 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.802520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:06.804052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:06.804441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:06.806695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:06.807868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:06.808687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:06.810162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:06.810467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e67d6-771f-460c-ba11-5c01a6b01503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-555e67d6-771f-460c-ba11-5c01a6b01503 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:06.818468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:06.819642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:08:06.820683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:06.833002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 360/713] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:08:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:06.833484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:06.837461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 354/714] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:06.838541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e67d6-771f-460c-ba11-5c01a6b01503 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:06.848284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:07.001844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e67d6-771f-460c-ba11-5c01a6b01503 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:07.005311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31524444-59b3-409e-80b6-28b272bee7a3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:07.021669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:07.053668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:07.056067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.056067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.056067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] PATH_INFO: `/v3/users/nova` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.063427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:07.065157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:07.067523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:07.068797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.069222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.069638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.070537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:07.070987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:07.072565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:07.081092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:07.082092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:07.083093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:07.086183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 355/715] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:08:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:07.095232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:07.095872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 361/716] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:08:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:07.104029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:07.218562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3abd36-26f9-45e4-9731-657a3b2c1f04 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:07.220735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:07.266579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:07.267857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.268364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.268948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.282387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a086dc7-fc4e-480d-aa84-df749c1a52af None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:07.284211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:07.286259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:07.318138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:07.319098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:07.320012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:07.320012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:07.320012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:07.320012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:07.325726 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 362/717] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:07.331389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:07.348858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:07.350696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.351367 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.351838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.352988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:07.353607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:07.355273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:07.383055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 356/718] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:08:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:07.387249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:07.389024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:07.466436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699da8a-c046-4cc5-a324-75d43db0cffd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:07.514646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:07.515358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.515640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.515910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.516644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:07.516946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:07.518350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:07.547973 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 363/719] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:07.556653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:07.597074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cf550-8ecb-44cf-86b6-d57bdd7b06cb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:07.614840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:07.632405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f1b7b-946e-4f0b-ae1b-93fea9cb8d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:07.633162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1b7b-946e-4f0b-ae1b-93fea9cb8d32 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.633518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1b7b-946e-4f0b-ae1b-93fea9cb8d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.633864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1b7b-946e-4f0b-ae1b-93fea9cb8d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.652155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:07.668909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aefd3-c5cb-4c8f-b0c1-0b558e2ab291 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:07.710091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f1b7b-946e-4f0b-ae1b-93fea9cb8d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:07.710091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f1b7b-946e-4f0b-ae1b-93fea9cb8d32 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:07.713210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d47bd-e212-4de5-a3e0-028125d1e6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:07.714241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d47bd-e212-4de5-a3e0-028125d1e6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.714702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d47bd-e212-4de5-a3e0-028125d1e6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.715116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d47bd-e212-4de5-a3e0-028125d1e6c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.723472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d47bd-e212-4de5-a3e0-028125d1e6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:07.725343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d47bd-e212-4de5-a3e0-028125d1e6c2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-267d47bd-e212-4de5-a3e0-028125d1e6c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:07.741328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:07.742385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:07.743378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:07.745702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 364/720] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:08:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:07.747532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43c9a-a4d7-4f66-bc93-ce6f43b7fa1e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.747985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43c9a-a4d7-4f66-bc93-ce6f43b7fa1e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.748404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43c9a-a4d7-4f66-bc93-ce6f43b7fa1e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.749838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 365/721] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:07.754827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f43c9a-a4d7-4f66-bc93-ce6f43b7fa1e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:07.818795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 357/722] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:08:07.820043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae6ba8-3e32-47a8-b013-35a5e1e01e98 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.820334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae6ba8-3e32-47a8-b013-35a5e1e01e98 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.820613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae6ba8-3e32-47a8-b013-35a5e1e01e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.837224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-22ae6ba8-3e32-47a8-b013-35a5e1e01e98 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:07.838418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-22ae6ba8-3e32-47a8-b013-35a5e1e01e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:07.852121 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:07.873239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f43c9a-a4d7-4f66-bc93-ce6f43b7fa1e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:07.908098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b49e6-b912-42b1-a729-5a56b24b9753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:07.908484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 358/723] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:08:07.908777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b49e6-b912-42b1-a729-5a56b24b9753 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.909062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b49e6-b912-42b1-a729-5a56b24b9753 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.909417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b49e6-b912-42b1-a729-5a56b24b9753 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.910260 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b49e6-b912-42b1-a729-5a56b24b9753 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:07.910733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b49e6-b912-42b1-a729-5a56b24b9753 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:07.911579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57906924-ddba-47cc-9184-999c5a38fd18 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.912062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57906924-ddba-47cc-9184-999c5a38fd18 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.912182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b49e6-b912-42b1-a729-5a56b24b9753 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:07.912693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57906924-ddba-47cc-9184-999c5a38fd18 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.914460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 359/724] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:07.915369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57906924-ddba-47cc-9184-999c5a38fd18 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:07.940187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 366/725] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 08:08:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:07.941622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9d919-20e0-4b56-8e5b-aaac0f0c6afe None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:07.941958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9d919-20e0-4b56-8e5b-aaac0f0c6afe None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:07.942239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9d919-20e0-4b56-8e5b-aaac0f0c6afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:07.949022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:07.960026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ccf9d919-20e0-4b56-8e5b-aaac0f0c6afe None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:07.961225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ccf9d919-20e0-4b56-8e5b-aaac0f0c6afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:08.022569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57906924-ddba-47cc-9184-999c5a38fd18 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.030062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 367/726] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:08:08.032182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9d919-20e0-4b56-8e5b-aaac0f0c6afe None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.059731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.060551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.060946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.061224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.072365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.074047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.083563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:08.094422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:08.095478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:08.096453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:08.096453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:08.096453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:08.096453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:08.103275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 360/727] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:08.103275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.145297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9d919-20e0-4b56-8e5b-aaac0f0c6afe None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.183539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbe12c-7643-4df4-9e27-006a507b9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.184352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe12c-7643-4df4-9e27-006a507b9eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.184663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe12c-7643-4df4-9e27-006a507b9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.185022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe12c-7643-4df4-9e27-006a507b9eb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.187947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:08.198542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbe12c-7643-4df4-9e27-006a507b9eb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:08.205534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b16410-3bb3-4be4-9ecc-6a681cc0c838 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4ecbe12c-7643-4df4-9e27-006a507b9eb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:08.206241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:08.207192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:08.208296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:08.212401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 368/728] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:08.213237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbe12c-7643-4df4-9e27-006a507b9eb2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.244857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8062b16-d7ba-49c5-9d86-9fc67bd92bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.246101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062b16-d7ba-49c5-9d86-9fc67bd92bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.246531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062b16-d7ba-49c5-9d86-9fc67bd92bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.247395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062b16-d7ba-49c5-9d86-9fc67bd92bd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.314025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:08.317414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8062b16-d7ba-49c5-9d86-9fc67bd92bd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.319584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8062b16-d7ba-49c5-9d86-9fc67bd92bd9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.320121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbe12c-7643-4df4-9e27-006a507b9eb2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.356982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40faa1c-200c-473b-b33a-73d010fe1a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.357602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40faa1c-200c-473b-b33a-73d010fe1a1d None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.357885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40faa1c-200c-473b-b33a-73d010fe1a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.358161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40faa1c-200c-473b-b33a-73d010fe1a1d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.358891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40faa1c-200c-473b-b33a-73d010fe1a1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.360345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40faa1c-200c-473b-b33a-73d010fe1a1d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.376653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 369/729] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:08:08 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:08.377612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40faa1c-200c-473b-b33a-73d010fe1a1d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.445066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 361/730] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 08:08:08.446674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8062b16-d7ba-49c5-9d86-9fc67bd92bd9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.462359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:08.532103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40faa1c-200c-473b-b33a-73d010fe1a1d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.570805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.571581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.571975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.572257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.572993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:08.573306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.574706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.576436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:08.579395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8062b16-d7ba-49c5-9d86-9fc67bd92bd9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.591907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 370/731] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:08.595188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.630207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.630886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.631170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.631584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.632338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:08.632644 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.634048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.642779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 362/732] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:08.643310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.720004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606e591-dd4b-4da4-b1c9-1fd26074acab None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.731201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:08.752765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab36e51-81ad-4fbc-87f5-73bdad62c238 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.759133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302038b-f128-4014-9e90-695a37899284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.759967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302038b-f128-4014-9e90-695a37899284 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.760249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302038b-f128-4014-9e90-695a37899284 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.760526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302038b-f128-4014-9e90-695a37899284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.771756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302038b-f128-4014-9e90-695a37899284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.773222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302038b-f128-4014-9e90-695a37899284 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.787982 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc4d3a-5327-4247-8dc6-8539a054ca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.788785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc4d3a-5327-4247-8dc6-8539a054ca75 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.789065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc4d3a-5327-4247-8dc6-8539a054ca75 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.789335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc4d3a-5327-4247-8dc6-8539a054ca75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4302038b-f128-4014-9e90-695a37899284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:08.791220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:08.792297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:08.793237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:08.793237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:08.793237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:08.793237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:08.793237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 371/733] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:08.793513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302038b-f128-4014-9e90-695a37899284 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.802317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc4d3a-5327-4247-8dc6-8539a054ca75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.803833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc4d3a-5327-4247-8dc6-8539a054ca75 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-bbbc4d3a-5327-4247-8dc6-8539a054ca75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:08.818614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:08.819754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:08.820686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 363/734] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:08.821185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3d0b-b9fb-4429-810e-1bdec33d6a1c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.821752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3d0b-b9fb-4429-810e-1bdec33d6a1c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.821752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3d0b-b9fb-4429-810e-1bdec33d6a1c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.823105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 364/735] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:08.823710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce3d0b-b9fb-4429-810e-1bdec33d6a1c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.849494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:08.910602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302038b-f128-4014-9e90-695a37899284 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.935939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce3d0b-b9fb-4429-810e-1bdec33d6a1c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:08.946524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.947355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.947638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.947913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.958969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.960476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.970940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddbf2f-f30e-46c0-9093-4e38aec2df13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:08.971630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddbf2f-f30e-46c0-9093-4e38aec2df13 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.971908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddbf2f-f30e-46c0-9093-4e38aec2df13 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.972176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddbf2f-f30e-46c0-9093-4e38aec2df13 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:08.972894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddbf2f-f30e-46c0-9093-4e38aec2df13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:08.973274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddbf2f-f30e-46c0-9093-4e38aec2df13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:08.974741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddbf2f-f30e-46c0-9093-4e38aec2df13 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:08.978758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:08.979673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:08.980570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:08.980570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:08.980570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:08.980570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:08.984665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 372/736] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:08.985466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:08.991031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 365/737] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:08.991031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb71de-358f-4549-be64-8f472711d747 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:08.991031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb71de-358f-4549-be64-8f472711d747 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:08.991031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb71de-358f-4549-be64-8f472711d747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.002771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:09.008445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5ddb71de-358f-4549-be64-8f472711d747 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:09.009965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5ddb71de-358f-4549-be64-8f472711d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:09.087236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 366/738] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 08:08:09.087236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb71de-358f-4549-be64-8f472711d747 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.100335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f9194-277f-4bca-9bb2-387f69d21cfc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:09.102024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:09.137471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:09.138134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:09.138416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:09.138692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.139518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:09.139827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:09.141268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:09.156528 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 373/739] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:09.157366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.196429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb71de-358f-4549-be64-8f472711d747 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:09.232041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:09.232735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:09.233018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:09.233289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.234016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:09.234320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:09.235728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:09.247562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 367/740] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:09.248154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.252747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:09.279383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94d27-8e37-4b6a-8850-62370b6c4378 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:09.314914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:09.315738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:09.316016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:09.316283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.321927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:09.323251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:09.324830 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:09.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:09.340163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 374/741] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:09.348559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.370068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6c51-f879-4109-984c-e2c96e942e89 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:09.405067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869638c-fab6-4335-8e26-ba0669ee0d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:09.405780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869638c-fab6-4335-8e26-ba0669ee0d5e None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:09.406062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869638c-fab6-4335-8e26-ba0669ee0d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:09.406336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869638c-fab6-4335-8e26-ba0669ee0d5e None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.407047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869638c-fab6-4335-8e26-ba0669ee0d5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:09.408493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869638c-fab6-4335-8e26-ba0669ee0d5e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:09.424708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 368/742] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:08:09 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:09.425473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869638c-fab6-4335-8e26-ba0669ee0d5e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.470179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83422d71-71ac-4970-aa81-3214da3e82b7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:09.520718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:09.527308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:09.528360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:09.528842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:09.529350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.530444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=96456bfed82d469b888bd495ec46e796) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:09.531058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:09.532861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:09.545839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 375/743] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:09 2022] GET /identity/v3/role_assignments?user.id=96456bfed82d469b888bd495ec46e796&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:09.547339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.573059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869638c-fab6-4335-8e26-ba0669ee0d5e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:09.610120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:09.617338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:09.618326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:09.619093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:09.619490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.633405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:09.637857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:09.661329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:09.662339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:08:09.663335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:09.664030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 369/744] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:09.676841 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.719608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f2b00-6295-4c93-88ef-52d65ce7cd98 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:09.771660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:09.773232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:09.773664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:09.774100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.775036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:09.775767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:09.778095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:09.797390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 376/745] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:09.803446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.819243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace3a12-4348-4fd6-ab35-ec6e95d42f14 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:09.855528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:09.856198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:09.856505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:09.857180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:09.858053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:09.861771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:09.861771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:09.887235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 370/746] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 08:08:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:09.887235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:09.927816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:09.945865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:10.002568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de016-7212-4d6c-9cb4-893a46d03e9b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:10.039931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b6132-3107-4873-a419-3b9c5d729668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:10.040821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b6132-3107-4873-a419-3b9c5d729668 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:10.041244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b6132-3107-4873-a419-3b9c5d729668 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:10.043161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b6132-3107-4873-a419-3b9c5d729668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:10.046091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8376f6-ebca-4639-a39e-ca393fcd476c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:10.119668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b6132-3107-4873-a419-3b9c5d729668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:10.121453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b6132-3107-4873-a419-3b9c5d729668 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:10.160597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:10.161273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:10.161557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:10.161831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:10.162675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=70d16599118b4968a18bf3f783a6687f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:10.162983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:10.164492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:10.182373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 377/747] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:09 2022] GET /identity/v3/role_assignments?user.id=70d16599118b4968a18bf3f783a6687f&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 259 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:10.186954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:10.247389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 371/748] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 08:08:10.253347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:10.371124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c506d-abc5-4f9d-be91-7be2be17f744 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:10.401286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:10.421795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880ee40-76f2-4e75-b7eb-c7494ad21532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:10.422886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880ee40-76f2-4e75-b7eb-c7494ad21532 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:10.423371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880ee40-76f2-4e75-b7eb-c7494ad21532 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:10.423794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880ee40-76f2-4e75-b7eb-c7494ad21532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:10.435913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880ee40-76f2-4e75-b7eb-c7494ad21532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:10.437595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880ee40-76f2-4e75-b7eb-c7494ad21532 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7880ee40-76f2-4e75-b7eb-c7494ad21532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:10.456814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:10.458658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:10.459821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:10.459821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:10.459821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:10.459821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:10.465949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 378/749] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:10.467540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b6132-3107-4873-a419-3b9c5d729668 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:10.574336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b6132-3107-4873-a419-3b9c5d729668 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:10.577207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d710a-5fe8-41d1-b52f-d6f9200f7da9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:10.577492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d710a-5fe8-41d1-b52f-d6f9200f7da9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:10.577769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d710a-5fe8-41d1-b52f-d6f9200f7da9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:10.579292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 379/750] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:10.584805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e059bc-a533-4514-81bd-6af56a19ff48 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:10.585361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e059bc-a533-4514-81bd-6af56a19ff48 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:10.585836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e059bc-a533-4514-81bd-6af56a19ff48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:10.603570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-97e059bc-a533-4514-81bd-6af56a19ff48 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:10.605140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-97e059bc-a533-4514-81bd-6af56a19ff48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:10.627563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:10.628567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:10.630538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:10.631055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:10.632008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:10.633574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:10.638340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:10.664253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 372/751] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:10.673291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:10.711655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:10.734151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 380/752] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:10.794064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e059bc-a533-4514-81bd-6af56a19ff48 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:10.980998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a3d5d-4579-4e72-978f-f192d6d4cf7e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:11.022001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:11.025205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:11.027153 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.027465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.027740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.028166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e059bc-a533-4514-81bd-6af56a19ff48 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:11.043600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:11.045073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:11.071255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:11.072207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:11.073073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:11.073073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:11.073073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:11.073073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:11.073073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:11.073073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:11.073073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:11.073073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:11.075447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 373/753] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:11.078417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:11.079254 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.079542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.079813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.080472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:11.081854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:11.086011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:11.108695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:11.109667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:11.110493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:11.120599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 381/754] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 23 08:08:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:11.121526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:11.176194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:11.210576 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa9fe-7c63-4208-a14c-21594b28e542 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:11.250088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:11.251639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.252271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.255072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.256107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:11.256618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:11.259267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:11.272102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 374/755] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:11.280178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:11.297952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b84741-1c2b-4f2c-9fdb-2681881d2150 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:11.332947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:11.333658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.333935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.334208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.334909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:11.335224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:11.336778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:11.341883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 382/756] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 08:08:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:11.347168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:11.381162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:11.463470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757300d-96f1-4dda-aad5-e0b1774e670f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:11.509815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:11.510995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.511506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.512386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.513308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:11.513811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:11.515444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:11.527617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 383/757] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 08:08:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:11.533729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:11.550026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2897ef5-09b1-4a24-9a67-114aff74931c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:11.587789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:11.596494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:11.597158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.597437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.597711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.599262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=7356a0f8d9f84e7cb785662db9afd658) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:11.599585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:11.601039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:11.622356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 375/758] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:11 2022] GET /identity/v3/role_assignments?user.id=7356a0f8d9f84e7cb785662db9afd658&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 259 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:11.687653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:11.697625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb59f5-9cef-42b0-910e-0dd8531b789f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:11.738056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b312d-7f34-4350-8ac6-38499feb312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:11.739529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b312d-7f34-4350-8ac6-38499feb312e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.739835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b312d-7f34-4350-8ac6-38499feb312e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.740117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b312d-7f34-4350-8ac6-38499feb312e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.740935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b312d-7f34-4350-8ac6-38499feb312e None admin] RBAC: Adding query filter params (interface=public, service_id=b84965995b7040f6b0ce8165c874f99a, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:11.741252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b312d-7f34-4350-8ac6-38499feb312e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:11.742673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b312d-7f34-4350-8ac6-38499feb312e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:11.755514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 384/759] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 08:08:11 2022] GET /identity/v3/endpoints?service_id=b84965995b7040f6b0ce8165c874f99a&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:11.758840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:11.829485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772bb86-bba7-4172-8c44-2db7a37aaa0a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.829822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772bb86-bba7-4172-8c44-2db7a37aaa0a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.830044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772bb86-bba7-4172-8c44-2db7a37aaa0a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.831770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 385/760] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:11.846135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b9deb-5e6a-4e88-b9c2-58dbca8252f4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.846451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b9deb-5e6a-4e88-b9c2-58dbca8252f4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.846720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b9deb-5e6a-4e88-b9c2-58dbca8252f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.869192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e26b9deb-5e6a-4e88-b9c2-58dbca8252f4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:11.870464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e26b9deb-5e6a-4e88-b9c2-58dbca8252f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:11.886003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb1da2-9a57-447d-9c29-dec1c5d5a94d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:11.938453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242e110-4604-47b4-b942-38e3b5b58499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:11.939128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e110-4604-47b4-b942-38e3b5b58499 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:11.940545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e110-4604-47b4-b942-38e3b5b58499 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:11.940824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e110-4604-47b4-b942-38e3b5b58499 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:11.941515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242e110-4604-47b4-b942-38e3b5b58499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:11.943107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242e110-4604-47b4-b942-38e3b5b58499 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:11.959305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 376/761] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:08:11 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:11.973703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 386/762] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:12.021240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242e110-4604-47b4-b942-38e3b5b58499 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:12.109015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:12.163761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242e110-4604-47b4-b942-38e3b5b58499 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:12.215652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:12.216464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.216747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.217022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] PATH_INFO: `/v3/services/network` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.217687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:12.219052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:12.235670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0787b-e2b5-4761-b037-6c7bc3f1345f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.236297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0787b-e2b5-4761-b037-6c7bc3f1345f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.236790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0787b-e2b5-4761-b037-6c7bc3f1345f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:12.236981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:12.237915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:12.238808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 377/763] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:12 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:12.243887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 387/764] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:12.244096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:12.249921 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d44925-f06c-4b74-a1fd-a5e5a453837a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.250474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d44925-f06c-4b74-a1fd-a5e5a453837a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.250945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d44925-f06c-4b74-a1fd-a5e5a453837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.270912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-93d44925-f06c-4b74-a1fd-a5e5a453837a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:12.272615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-93d44925-f06c-4b74-a1fd-a5e5a453837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:12.367153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 388/765] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:12.376768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91004c7e-0dcf-4de2-a569-a5f1dbb0acce None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:12.395283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d44925-f06c-4b74-a1fd-a5e5a453837a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:12.438474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:12.439528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:12.459472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:12.460153 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.460431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.460701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.461418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] RBAC: Adding query filter params (name=network) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:12.461720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:12.463160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:12.474943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 378/766] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:12.477996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:12.597455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d44925-f06c-4b74-a1fd-a5e5a453837a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:12.644980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80118-fb4a-44ab-a416-ca0d8beb17af None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:12.649022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974201f-be14-4b0c-8811-7028bf010108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:12.649929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974201f-be14-4b0c-8811-7028bf010108 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.650953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974201f-be14-4b0c-8811-7028bf010108 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.650953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974201f-be14-4b0c-8811-7028bf010108 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.661838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974201f-be14-4b0c-8811-7028bf010108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f974201f-be14-4b0c-8811-7028bf010108 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:12.669720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:12.670833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:12.671890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:12.678439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 389/767] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:12.684262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974201f-be14-4b0c-8811-7028bf010108 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:12.702866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:12.704032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.704506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.704968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.705871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] RBAC: Adding query filter params (type=network) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:12.706379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:12.708132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:12.716700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 379/768] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:12.723478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:12.791612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:12.800190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:12.824248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974201f-be14-4b0c-8811-7028bf010108 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:12.852581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a05c6-5b34-4402-b50f-d0f401649f93 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:12.866930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91a15-0c60-4792-84fb-6bcc79176e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:12.868028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91a15-0c60-4792-84fb-6bcc79176e39 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.868565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91a15-0c60-4792-84fb-6bcc79176e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.868846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91a15-0c60-4792-84fb-6bcc79176e39 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.869787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91a15-0c60-4792-84fb-6bcc79176e39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:12.870101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91a15-0c60-4792-84fb-6bcc79176e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:12.872132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91a15-0c60-4792-84fb-6bcc79176e39 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:12.882879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 390/769] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:12.884437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be5977-bceb-43ba-a4b8-e8b3078fd598 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.884718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be5977-bceb-43ba-a4b8-e8b3078fd598 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.884987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be5977-bceb-43ba-a4b8-e8b3078fd598 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.890511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 391/770] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:12.891412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be5977-bceb-43ba-a4b8-e8b3078fd598 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:12.895574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd89a55-41f7-4e3e-a8fc-b1128760b188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:12.896251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd89a55-41f7-4e3e-a8fc-b1128760b188 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.896539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd89a55-41f7-4e3e-a8fc-b1128760b188 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.896815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd89a55-41f7-4e3e-a8fc-b1128760b188 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.897615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd89a55-41f7-4e3e-a8fc-b1128760b188 None admin] RBAC: Adding query filter params (interface=public, service_id=1a873d2badb745269fb6f0e12ecd1908, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:12.897930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd89a55-41f7-4e3e-a8fc-b1128760b188 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:12.899351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd89a55-41f7-4e3e-a8fc-b1128760b188 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:12.908401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 380/771] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 08:08:12 2022] GET /identity/v3/endpoints?service_id=1a873d2badb745269fb6f0e12ecd1908&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:12.909736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a26f8-cf50-4b20-beca-2e5b22b07453 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:12.910026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a26f8-cf50-4b20-beca-2e5b22b07453 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:12.910469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a26f8-cf50-4b20-beca-2e5b22b07453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:12.938923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ec2a26f8-cf50-4b20-beca-2e5b22b07453 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:12.940720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ec2a26f8-cf50-4b20-beca-2e5b22b07453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:13.013844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be5977-bceb-43ba-a4b8-e8b3078fd598 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.018215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 381/772] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:13.019419 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a26f8-cf50-4b20-beca-2e5b22b07453 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.050248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.051143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.051454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.051761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.059536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:13.064580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:13.066625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:13.069426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:13.087089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:13.088228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:13.089323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:13.089323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:13.089323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:13.089323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:13.089815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 392/773] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:13.090812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.127745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a26f8-cf50-4b20-beca-2e5b22b07453 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.161588 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76611c69-7d27-4b1f-821d-0ed9f53ae592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.162459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611c69-7d27-4b1f-821d-0ed9f53ae592 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.162968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611c69-7d27-4b1f-821d-0ed9f53ae592 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.163474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611c69-7d27-4b1f-821d-0ed9f53ae592 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.164364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76611c69-7d27-4b1f-821d-0ed9f53ae592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:13.165913 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76611c69-7d27-4b1f-821d-0ed9f53ae592 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:13.179355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 382/774] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/services => generated 772 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:13.180716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76611c69-7d27-4b1f-821d-0ed9f53ae592 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.203538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce35277-d3a7-44c8-ac32-2ade1de0ef45 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.247232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef0618-7a9b-4703-a581-11111802518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.247232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef0618-7a9b-4703-a581-11111802518f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.247232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef0618-7a9b-4703-a581-11111802518f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.247232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef0618-7a9b-4703-a581-11111802518f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.251834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef0618-7a9b-4703-a581-11111802518f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-31ef0618-7a9b-4703-a581-11111802518f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:13.261793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:13.262761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:13.263638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 393/775] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:13.276960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef0618-7a9b-4703-a581-11111802518f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.322244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76611c69-7d27-4b1f-821d-0ed9f53ae592 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.335593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:13.363529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.364677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.365361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.367161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.368178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:13.369464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:13.371103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:13.395887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 383/776] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:13.404488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.413150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:13.521588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef0618-7a9b-4703-a581-11111802518f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.562443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.563291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.563573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.563847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.564832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:13.565143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:13.566969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:13.576242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 394/777] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:13.581934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.598588 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429933ad-e048-496c-9692-378b7ab3528f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.658655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.659442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.659721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.659996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.668833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:13.670397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:13.685469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:13.686443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:13.687431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 384/778] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:13.692745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.743653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:13.746818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235f1c7-e21c-466f-bf7b-0bbe34d56b75 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.764094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:13.784211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.785024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.785425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.786280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.797223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:13.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:13.818949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:13.821097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:13.823167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:13.823167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:13.823167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:13.823167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:13.823167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 395/779] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:13.827876 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.899235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869aeb2-629b-4b34-bfd5-edd78e4d5010 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.939419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a452e2-75dc-4196-9dab-bf01cb7d18d5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:13.951610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af1af7-9570-475b-9a41-b65d245985d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.952285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af1af7-9570-475b-9a41-b65d245985d0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.952565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af1af7-9570-475b-9a41-b65d245985d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.952839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af1af7-9570-475b-9a41-b65d245985d0 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.953773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af1af7-9570-475b-9a41-b65d245985d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:13.954085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af1af7-9570-475b-9a41-b65d245985d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:13.955729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af1af7-9570-475b-9a41-b65d245985d0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:13.975470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 385/780] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:13.976466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bff50-899a-49dd-b531-fce315d4380d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:13.976884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b9101-967c-4cbf-bd36-1402e50f191c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.977109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bff50-899a-49dd-b531-fce315d4380d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.977238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b9101-967c-4cbf-bd36-1402e50f191c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.977416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bff50-899a-49dd-b531-fce315d4380d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.977509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b9101-967c-4cbf-bd36-1402e50f191c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.977670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bff50-899a-49dd-b531-fce315d4380d None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:13.978408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bff50-899a-49dd-b531-fce315d4380d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:13.978714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bff50-899a-49dd-b531-fce315d4380d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:13.980157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bff50-899a-49dd-b531-fce315d4380d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:13.982997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 386/781] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:13.984021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b9101-967c-4cbf-bd36-1402e50f191c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:13.994846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 396/782] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:13.996183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892773c-9ef5-4ac2-aa45-25bc8932e352 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:13.996471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892773c-9ef5-4ac2-aa45-25bc8932e352 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:13.997058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892773c-9ef5-4ac2-aa45-25bc8932e352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.013907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3892773c-9ef5-4ac2-aa45-25bc8932e352 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:14.015117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3892773c-9ef5-4ac2-aa45-25bc8932e352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:14.034583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:14.086251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:14.095229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 397/783] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:14.095229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892773c-9ef5-4ac2-aa45-25bc8932e352 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.119905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b9101-967c-4cbf-bd36-1402e50f191c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.174119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.176975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.177281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.177524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.193830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:14.196591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:14.215035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892773c-9ef5-4ac2-aa45-25bc8932e352 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:14.223723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:14.224620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:14.225523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:14.225523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:14.225523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:14.225523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:14.228628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 387/784] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:14.229893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.253657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.254799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.254986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.255271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.263613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:14.265271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:14.280499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:14.281478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:14.282426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 398/785] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:14.283011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.301020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:14.345626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60171e-ce9e-4e6e-92cf-232abe4f8c62 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.383118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb3a4-6105-4ffc-9437-c9b5b7880df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.384386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb3a4-6105-4ffc-9437-c9b5b7880df8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.384915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb3a4-6105-4ffc-9437-c9b5b7880df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.385395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb3a4-6105-4ffc-9437-c9b5b7880df8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.392738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466dc5-4b2b-4507-9365-171e79fc9e79 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.394126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:14.400296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfb3a4-6105-4ffc-9437-c9b5b7880df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0cbfb3a4-6105-4ffc-9437-c9b5b7880df8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:14.408358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:14.409268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:14.410124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:14.413595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 388/786] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:14.414615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb3a4-6105-4ffc-9437-c9b5b7880df8 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.437393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.438126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.438353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.438634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.439839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:14.440337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:14.441743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:14.453863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 399/787] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:14.454505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.530796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb3a4-6105-4ffc-9437-c9b5b7880df8 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.541136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:14.569668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221d1cd-edf9-4b8a-a8cd-46b9f219f9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.570349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221d1cd-edf9-4b8a-a8cd-46b9f219f9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.570629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221d1cd-edf9-4b8a-a8cd-46b9f219f9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.570904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221d1cd-edf9-4b8a-a8cd-46b9f219f9f3 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.571089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aedf4-5491-4989-ab3c-609df9b6399a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.571785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221d1cd-edf9-4b8a-a8cd-46b9f219f9f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:14.572096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221d1cd-edf9-4b8a-a8cd-46b9f219f9f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:14.573535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221d1cd-edf9-4b8a-a8cd-46b9f219f9f3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:14.591755 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 389/788] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:14.593055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f378070-ed78-4756-88ae-e8f9c86d41c9 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.593435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f378070-ed78-4756-88ae-e8f9c86d41c9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.593719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f378070-ed78-4756-88ae-e8f9c86d41c9 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.599311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 390/789] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:14.600132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f378070-ed78-4756-88ae-e8f9c86d41c9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.612701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.613742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.614206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.614653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.615614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:14.616110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:14.617509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:14.626790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 400/790] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:14.627689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.665621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:14.716282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f378070-ed78-4756-88ae-e8f9c86d41c9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.730863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8375c4-48c5-48ba-b111-d07376d5334e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.751591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7999bcf-40ad-45cd-b3aa-6f74733c61af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.752397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999bcf-40ad-45cd-b3aa-6f74733c61af None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.752676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999bcf-40ad-45cd-b3aa-6f74733c61af None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.753078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999bcf-40ad-45cd-b3aa-6f74733c61af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.759907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7999bcf-40ad-45cd-b3aa-6f74733c61af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:14.761353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7999bcf-40ad-45cd-b3aa-6f74733c61af None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:14.771441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.772238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.772516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.772783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c7999bcf-40ad-45cd-b3aa-6f74733c61af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:14.775392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:14.776249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:14.777131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:14.777131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:14.777131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:14.777131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:14.777131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:14.777131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:14.777131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:14.777131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:14.777585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:14.778025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 391/791] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:14.778025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a694-8572-43e7-b9a4-0b5363f40acc None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.778025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a694-8572-43e7-b9a4-0b5363f40acc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.778260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a694-8572-43e7-b9a4-0b5363f40acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.788727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:14.790217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:14.797135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ecf0a694-8572-43e7-b9a4-0b5363f40acc None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:14.798321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ecf0a694-8572-43e7-b9a4-0b5363f40acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:14.808330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:14.809354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:14.810252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:14.810252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:14.810252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:14.810252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:14.811346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 401/792] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:14.812342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.874634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 392/793] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:14.875507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a694-8572-43e7-b9a4-0b5363f40acc None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:14.922850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:14.931526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f039ec2-6dff-4101-90b8-cd33ce55fabd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.966447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:14.967171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:14.967480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:14.967754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:14.977336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a694-8572-43e7-b9a4-0b5363f40acc None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:14.978835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:14.980352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:15.000924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:15.001879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:15.002804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:15.002804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:15.002804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:15.002804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:15.003025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 402/794] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:15.003840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.017126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.017793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.018070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.018344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.019160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:15.019500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.021006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.032712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 393/795] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:15.033499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.057600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:15.120984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc101-ae55-46fb-ba3d-77c9b79d8c5f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.139099 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa6476-343b-4b63-b7a4-e835ff6959b5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.146396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:15.158831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.159535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.159822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.160465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.160927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:15.161230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.162618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.175117 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.180025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 403/796] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:15.180164 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.180164 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.180321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.180402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.180402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.180402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:15.192483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:15.193494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:15.194328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 394/797] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:15.195201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.290413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:15.297665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266248-cd75-4cf8-a35c-3a4e60c54cf3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.309092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fec2a-eb88-4077-a424-7d9dc2a4d48b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.336984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.337656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.337949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.338199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.338984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:15.339244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.340650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.344943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70372d4a-f1a0-4088-9032-d26649c60d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.345846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70372d4a-f1a0-4088-9032-d26649c60d2f None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.346126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70372d4a-f1a0-4088-9032-d26649c60d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.346396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70372d4a-f1a0-4088-9032-d26649c60d2f None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/96456bfed82d469b888bd495ec46e796/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.348591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70372d4a-f1a0-4088-9032-d26649c60d2f None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=96456bfed82d469b888bd495ec46e796, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.350004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70372d4a-f1a0-4088-9032-d26649c60d2f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.356754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 404/798] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:15.356754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.379235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 395/799] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:08:15 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/96456bfed82d469b888bd495ec46e796/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:08:15.379235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70372d4a-f1a0-4088-9032-d26649c60d2f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.411361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:15.495723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70372d4a-f1a0-4088-9032-d26649c60d2f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.513482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa7cb0-b0b3-4d01-9958-93d643ecacf1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.537301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.537979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.538257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.538530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.539376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:15.539684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.541253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.551431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 396/800] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:15.551431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.558535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bacfe-3afb-4ff1-97af-8278e9b6d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.559388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bacfe-3afb-4ff1-97af-8278e9b6d468 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.559668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bacfe-3afb-4ff1-97af-8278e9b6d468 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.559934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bacfe-3afb-4ff1-97af-8278e9b6d468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.567476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bacfe-3afb-4ff1-97af-8278e9b6d468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.568952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bacfe-3afb-4ff1-97af-8278e9b6d468 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0e5bacfe-3afb-4ff1-97af-8278e9b6d468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:15.583276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:15.584362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:15.585283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 405/801] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:15.585846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8ec7a-a0a4-4e72-b9a2-a5633d0a1c81 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.586161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8ec7a-a0a4-4e72-b9a2-a5633d0a1c81 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.586740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8ec7a-a0a4-4e72-b9a2-a5633d0a1c81 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.587808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 406/802] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:15.588405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8ec7a-a0a4-4e72-b9a2-a5633d0a1c81 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.611855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:15.652481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:15.659596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68ef0f-87fc-4215-9efb-83796a37ed1c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.692973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.693777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.694059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.694340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.701919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.702465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8ec7a-a0a4-4e72-b9a2-a5633d0a1c81 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.703383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:15.718006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:15.719002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 397/803] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:15.720011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.750062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229538-38ce-4387-8555-a673ef2a6b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.750729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229538-38ce-4387-8555-a673ef2a6b20 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.751007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229538-38ce-4387-8555-a673ef2a6b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.751307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229538-38ce-4387-8555-a673ef2a6b20 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.752016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae229538-38ce-4387-8555-a673ef2a6b20 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:15.752319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae229538-38ce-4387-8555-a673ef2a6b20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.753761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae229538-38ce-4387-8555-a673ef2a6b20 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.759316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 407/804] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:15.765739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba3d0-94ae-4ffb-801d-b7ad47e4a23b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.766024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba3d0-94ae-4ffb-801d-b7ad47e4a23b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.766323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba3d0-94ae-4ffb-801d-b7ad47e4a23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.784300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cb1ba3d0-94ae-4ffb-801d-b7ad47e4a23b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:15.785584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cb1ba3d0-94ae-4ffb-801d-b7ad47e4a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:15.829612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72891095-c3e4-4633-b81f-80dd4d196b62 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:15.859298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 408/805] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:15.860144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba3d0-94ae-4ffb-801d-b7ad47e4a23b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.876873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:15.877471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:15.879175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:15.879693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:15.880181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:15.881877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:15.882475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:15.884145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:15.898502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 398/806] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:15.899664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:15.918912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:15.965540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba3d0-94ae-4ffb-801d-b7ad47e4a23b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.002485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.003156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.003466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.003742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.004735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:16.005149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.006602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.007847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f33c-c7db-4d8f-95ab-f328ad5a91dd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.030026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 409/807] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 08:08:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:16.030690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.046053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2d1f0-d545-4c63-b50f-99de821bbdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.046707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2d1f0-d545-4c63-b50f-99de821bbdb2 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.046981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2d1f0-d545-4c63-b50f-99de821bbdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.047274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2d1f0-d545-4c63-b50f-99de821bbdb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.048043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2d1f0-d545-4c63-b50f-99de821bbdb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.049386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2d1f0-d545-4c63-b50f-99de821bbdb2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.083326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 399/808] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:08:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:16.083326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2d1f0-d545-4c63-b50f-99de821bbdb2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.107188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:16.144078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2b49a-f3d4-4a01-b5df-0464a1fdcd87 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.183583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:16.194755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.195892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.196418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.196857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] PATH_INFO: `/v3/services/network` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.197642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.199302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:16.213573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:16.214817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 08:08:16.215844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:16.218360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 410/809] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:16 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:16.219584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.223714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2d1f0-d545-4c63-b50f-99de821bbdb2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.260813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8b96-77dc-4d10-9e47-ea111e344047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.262635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8b96-77dc-4d10-9e47-ea111e344047 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.263046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8b96-77dc-4d10-9e47-ea111e344047 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.263454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8b96-77dc-4d10-9e47-ea111e344047 None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/70d16599118b4968a18bf3f783a6687f/roles/fa8c35ad77b442fbbc0d20d458fe8731` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.280516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e8b96-77dc-4d10-9e47-ea111e344047 None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=70d16599118b4968a18bf3f783a6687f, role_id=fa8c35ad77b442fbbc0d20d458fe8731)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.282763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e8b96-77dc-4d10-9e47-ea111e344047 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.298653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 400/810] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:08:16 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/70d16599118b4968a18bf3f783a6687f/roles/fa8c35ad77b442fbbc0d20d458fe8731 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:08:16.299310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8b96-77dc-4d10-9e47-ea111e344047 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.338282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96e90-d52a-41e3-b46a-e30d592dc6bf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.380468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:16.388132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.388971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.389258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.390449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.405337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.407032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.425404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8b96-77dc-4d10-9e47-ea111e344047 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:16.430774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:16.431768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:16.432629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:16.432629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:16.432629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:16.432629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:16.432629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 411/811] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:16.438720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.452511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:16.487151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.487932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.488229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.488518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.489514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:16.489514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.492529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.502703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 401/812] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:16.515484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.570699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14aec4-ce0c-47b5-9017-07199fdb8c50 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.606980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.607925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.608352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.608768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.609632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:16.610083 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.611678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.624805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 412/813] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:16.630541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.684901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:16.736684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7459861-0f4d-4c09-b266-75b59439cdf4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.757211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88773f38-eee0-440f-89b2-baa0b308b9c9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.774857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.775951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.776413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.776829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.784900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.786876 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:16.802170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:16.803324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:16.804272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:16.804272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:16.804272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:16.804272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:16.804272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:16.804272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:16.804272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:16.804272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:16.806902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 413/814] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:16.813085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.866063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:16.866750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:16.867030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:16.867336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:16.868047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] RBAC: Adding query filter params (type=network) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:16.868355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:16.869729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:16.877950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 402/815] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:16.892232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:16.975784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e394a7-f73a-4b32-bbd3-880691b0b175 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:16.977508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:16.987715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:17.044945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:17.045670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:17.045956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:17.046235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:17.047033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:17.047370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:17.048833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:17.070079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 414/816] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:17.072140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb8461-30c4-4eb0-90e5-fef671c2d272 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:17.077729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:17.132971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01485ec6-50df-4b44-b65a-8e99d92159e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:17.133562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01485ec6-50df-4b44-b65a-8e99d92159e7 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:17.133845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01485ec6-50df-4b44-b65a-8e99d92159e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:17.134126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01485ec6-50df-4b44-b65a-8e99d92159e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:17.136366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01485ec6-50df-4b44-b65a-8e99d92159e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:17.137759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01485ec6-50df-4b44-b65a-8e99d92159e7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:17.172244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 403/817] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:08:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:08:17.238122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79074b51-8160-4ed5-8310-8d5cbe6d6202 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:17.296196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f56ebd-333a-471a-bf0c-bfb997d9769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:17.297135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f56ebd-333a-471a-bf0c-bfb997d9769c None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:17.297414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f56ebd-333a-471a-bf0c-bfb997d9769c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:17.297688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f56ebd-333a-471a-bf0c-bfb997d9769c None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/7356a0f8d9f84e7cb785662db9afd658/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:17.299860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f56ebd-333a-471a-bf0c-bfb997d9769c None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=7356a0f8d9f84e7cb785662db9afd658, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:17.301606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f56ebd-333a-471a-bf0c-bfb997d9769c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:17.326337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 415/818] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:08:17 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/7356a0f8d9f84e7cb785662db9afd658/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:08:17.672508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaf885-b190-45ba-b000-8f8da8a8493b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:17.672909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaf885-b190-45ba-b000-8f8da8a8493b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:17.673088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaf885-b190-45ba-b000-8f8da8a8493b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:17.674718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 404/819] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:17.680744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309acdd-5764-44ba-8930-3a4d0846c12f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:17.681214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309acdd-5764-44ba-8930-3a4d0846c12f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:17.681291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309acdd-5764-44ba-8930-3a4d0846c12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:17.698606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f309acdd-5764-44ba-8930-3a4d0846c12f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:17.700134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f309acdd-5764-44ba-8930-3a4d0846c12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:17.715711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:17.773947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 416/820] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:17.797871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaf885-b190-45ba-b000-8f8da8a8493b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:17.906271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaf885-b190-45ba-b000-8f8da8a8493b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:17.909120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:17.942845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c964078-4b5a-45f5-bc58-d610b824704b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:17.943681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c964078-4b5a-45f5-bc58-d610b824704b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:17.943965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c964078-4b5a-45f5-bc58-d610b824704b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:17.944241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c964078-4b5a-45f5-bc58-d610b824704b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:17.954944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c964078-4b5a-45f5-bc58-d610b824704b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-1c964078-4b5a-45f5-bc58-d610b824704b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:17.963250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:17.964181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:17.965018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 405/821] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:17.970654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309acdd-5764-44ba-8930-3a4d0846c12f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:18.071269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309acdd-5764-44ba-8930-3a4d0846c12f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:18.105661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:18.106341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.106626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.106901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.107709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:18.107985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:18.109394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:18.118567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 417/822] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:18.124399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c964078-4b5a-45f5-bc58-d610b824704b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:18.231226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c964078-4b5a-45f5-bc58-d610b824704b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:18.266166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:18.267375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.267850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.268308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.279545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:18.281257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:18.299226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:18.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:18.301371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:18.301371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:18.301371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:18.301371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:18.303657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 406/823] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:18.311867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:18.331671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:18.419098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b23a81-9e74-4890-bf82-0cb5573c06c9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:18.453431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd670f23-e373-4b8f-8f1c-8eb84dac0c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:18.454340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670f23-e373-4b8f-8f1c-8eb84dac0c80 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.454788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670f23-e373-4b8f-8f1c-8eb84dac0c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.455247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670f23-e373-4b8f-8f1c-8eb84dac0c80 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.456103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd670f23-e373-4b8f-8f1c-8eb84dac0c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:18.456546 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd670f23-e373-4b8f-8f1c-8eb84dac0c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:18.458774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd670f23-e373-4b8f-8f1c-8eb84dac0c80 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:18.471278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 418/824] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:18.480524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:18.504274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:18.533716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbba17-15fb-4746-8e1b-3fdd573ff183 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.534050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbba17-15fb-4746-8e1b-3fdd573ff183 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.534274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbba17-15fb-4746-8e1b-3fdd573ff183 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.535993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 419/825] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:18.541204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9fa80-9470-4635-abe2-b9095a6a340e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.541480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9fa80-9470-4635-abe2-b9095a6a340e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.541764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9fa80-9470-4635-abe2-b9095a6a340e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.559353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c2c9fa80-9470-4635-abe2-b9095a6a340e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:18.560595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c2c9fa80-9470-4635-abe2-b9095a6a340e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:18.612333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14726f8f-dd4d-4977-97c8-4d80ad934e65 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:18.635545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 420/826] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:18.646267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eefd9-7799-441b-a153-0e5cb500dda1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.646559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eefd9-7799-441b-a153-0e5cb500dda1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.646848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eefd9-7799-441b-a153-0e5cb500dda1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.648451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 421/827] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:18.650111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:18.651257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.651729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.652434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.653412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff847b-d3e6-47ba-b446-54fe8f366ca1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.653792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff847b-d3e6-47ba-b446-54fe8f366ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.654070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff847b-d3e6-47ba-b446-54fe8f366ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.659237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:18.661813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:18.663719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:18.672858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d7ff847b-d3e6-47ba-b446-54fe8f366ca1 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:18.674036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d7ff847b-d3e6-47ba-b446-54fe8f366ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:18.678473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:18.679668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:18.680604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 407/828] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:08:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:18.681185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:18.747507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 422/829] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:18.748359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff847b-d3e6-47ba-b446-54fe8f366ca1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:18.773697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:18.798316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a2799-4c1d-4ade-abe1-1f21e6aafeaf None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:18.832992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:18.833800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.834084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.834360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.835024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:18.836473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:18.849600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:18.850597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:18.851480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 408/830] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:18.852307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:18.865852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff847b-d3e6-47ba-b446-54fe8f366ca1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:18.898482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:18.905155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:18.905837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.906112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.906387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:18.907255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:18.907545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:18.908987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:18.932142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 423/831] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:08:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:18.932142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:18.961521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f1fe-6454-4d3d-8806-b6f325d138a2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:18.996090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5291c9e-d206-4172-a67b-965596c4a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:18.997347 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5291c9e-d206-4172-a67b-965596c4a80a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:18.997632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5291c9e-d206-4172-a67b-965596c4a80a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:18.997909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5291c9e-d206-4172-a67b-965596c4a80a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.005549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:19.008816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5291c9e-d206-4172-a67b-965596c4a80a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d5291c9e-d206-4172-a67b-965596c4a80a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:19.016872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:19.017791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:19.018698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 409/832] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:08:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:19.019425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5291c9e-d206-4172-a67b-965596c4a80a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.046092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e1f0-5136-406c-9461-349fe317424a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.085692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.086360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.086639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.086911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.087642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:19.087949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.089500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.100441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 424/833] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:19.100441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.128531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5291c9e-d206-4172-a67b-965596c4a80a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.159202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:19.165753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b6c3-7d69-4524-8eb9-2548661d1e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.166408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b6c3-7d69-4524-8eb9-2548661d1e9d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.166684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b6c3-7d69-4524-8eb9-2548661d1e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.167112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b6c3-7d69-4524-8eb9-2548661d1e9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.205736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acd39a-261e-4891-9e28-c1c10e1c0bbe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.233850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:19.236208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4b6c3-7d69-4524-8eb9-2548661d1e9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.237937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4b6c3-7d69-4524-8eb9-2548661d1e9d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.245814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.246794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.247073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.247373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.248122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:19.248426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.250210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.260212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 425/834] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:19.265499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.343431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 410/835] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:08:19.344589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b6c3-7d69-4524-8eb9-2548661d1e9d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.376625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6551d-bd99-42ca-bb1e-eadcbd5b0cba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.396039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:19.415145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.415857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.416140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.416420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.417131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:19.417441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.418832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.424236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 426/836] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:19.425105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.453278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:19.457804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b6c3-7d69-4524-8eb9-2548661d1e9d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.497935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4215927-8dda-4f53-8fbf-4eaa1414f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.498759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4215927-8dda-4f53-8fbf-4eaa1414f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.499043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4215927-8dda-4f53-8fbf-4eaa1414f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.499352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4215927-8dda-4f53-8fbf-4eaa1414f5f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.513988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4215927-8dda-4f53-8fbf-4eaa1414f5f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.515556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4215927-8dda-4f53-8fbf-4eaa1414f5f4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e4215927-8dda-4f53-8fbf-4eaa1414f5f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:19.534835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:19.535948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:19.537635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:19.537635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:19.537635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:19.537635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:19.537635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 411/837] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:19.538403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255adb61-9f07-40ff-9904-76feb0cc7a41 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.538521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255adb61-9f07-40ff-9904-76feb0cc7a41 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.538713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255adb61-9f07-40ff-9904-76feb0cc7a41 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.540384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 412/838] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:19.543327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255adb61-9f07-40ff-9904-76feb0cc7a41 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.555884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19bd08-a9ec-4d1b-b4c7-762176ce87a4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.596262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d581f-5c7b-47c3-badc-43a2e2f59657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.597342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d581f-5c7b-47c3-badc-43a2e2f59657 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.597342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d581f-5c7b-47c3-badc-43a2e2f59657 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.597714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d581f-5c7b-47c3-badc-43a2e2f59657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.610066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d581f-5c7b-47c3-badc-43a2e2f59657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.611562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d581f-5c7b-47c3-badc-43a2e2f59657 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1c2d581f-5c7b-47c3-badc-43a2e2f59657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:19.629164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:19.630089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:19.635960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:19.635960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:19.635960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:19.635960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:19.635960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 427/839] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:19.635960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c064680-b70f-4837-bb84-3d0a1128581c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.635960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c064680-b70f-4837-bb84-3d0a1128581c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.635960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c064680-b70f-4837-bb84-3d0a1128581c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.648867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9c064680-b70f-4837-bb84-3d0a1128581c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:19.649969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9c064680-b70f-4837-bb84-3d0a1128581c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:19.669240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255adb61-9f07-40ff-9904-76feb0cc7a41 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.676709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:19.716395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.717024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.717348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.717611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.718282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:19.718584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.719977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.721464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 428/840] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:19.722061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c064680-b70f-4837-bb84-3d0a1128581c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.735253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 413/841] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:19.738981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.758360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:19.830322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c064680-b70f-4837-bb84-3d0a1128581c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.850948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f78044-3334-472f-bac9-722bdfc97156 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:19.866767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.867469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.867750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.868022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.868749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:19.869056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.870458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.882148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 429/842] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:19.882956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.892928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:19.893961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:19.894715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:19.894973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:19.903766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:19.905243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:19.910952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:19.922875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:19.923921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 414/843] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:19.929743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:19.998885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c8499-7f4e-4f67-a6e8-46cb9b0682bf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.004623 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:20.033993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e651c-4647-4f05-b0d7-91cd9637455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.034803 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e651c-4647-4f05-b0d7-91cd9637455c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.035115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e651c-4647-4f05-b0d7-91cd9637455c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.035419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e651c-4647-4f05-b0d7-91cd9637455c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.038213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678402b-df8d-44aa-8140-961aeb4ffa47 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.046109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e651c-4647-4f05-b0d7-91cd9637455c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-400e651c-4647-4f05-b0d7-91cd9637455c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:20.053874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:20.054815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:20.055719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 430/844] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:20.056426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e651c-4647-4f05-b0d7-91cd9637455c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.080661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.081464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.081740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.082047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.089072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.090506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:20.106986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:20.108150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:20.109282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:20.109282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:20.109282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:20.109282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:20.109282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:20.109282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:20.109282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:20.109282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:20.112922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 415/845] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:20.114033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.152324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:20.171952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e651c-4647-4f05-b0d7-91cd9637455c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.208497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.209318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.209693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.210068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.210968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:20.211670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.213122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.219680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace85187-3f63-48b5-9dc7-11899470f540 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.230642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 431/846] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:20.231770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.251483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:20.258480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.259393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.259735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.260104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.260929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:20.261238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.262723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.275560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 416/847] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:20.276692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.340204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7e53-0144-4c62-a464-ea42217004de None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.377183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.377826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.378105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.378375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.379363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:20.379939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.381219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.382474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ebb9-2aa6-4469-aa11-a73a40261430 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.390699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:20.394615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 432/848] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:20.395466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.427472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8cee-ea35-4d56-b52b-2de18c7b3224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.428502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8cee-ea35-4d56-b52b-2de18c7b3224 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.429129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8cee-ea35-4d56-b52b-2de18c7b3224 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.429575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8cee-ea35-4d56-b52b-2de18c7b3224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.498846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:20.504489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb12344-7b3b-42c2-a692-da21dfb0681a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.512702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb8cee-ea35-4d56-b52b-2de18c7b3224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.514168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb8cee-ea35-4d56-b52b-2de18c7b3224 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.544716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.545518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.545799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.546080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.558218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.560015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:20.577540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:20.578657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:20.579907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:20.579907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:20.579907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:20.579907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:20.579907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 433/849] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:20.584717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.622917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 417/850] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:08:20.624033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8cee-ea35-4d56-b52b-2de18c7b3224 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.640435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:20.690742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2f6e0-5d60-4762-9d74-cfed0128bc14 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.726253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4022102-422c-4987-8255-52733213248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.726729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:20.726906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4022102-422c-4987-8255-52733213248d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.727193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4022102-422c-4987-8255-52733213248d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.727499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4022102-422c-4987-8255-52733213248d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.728591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4022102-422c-4987-8255-52733213248d None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=96456bfed82d469b888bd495ec46e796) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:20.728739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4022102-422c-4987-8255-52733213248d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.730115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4022102-422c-4987-8255-52733213248d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.734422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8cee-ea35-4d56-b52b-2de18c7b3224 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.743248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 434/851] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/role_assignments?user.id=96456bfed82d469b888bd495ec46e796&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:20.743248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4022102-422c-4987-8255-52733213248d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.788456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.789320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.789896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.790177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.790916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:20.791240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.793272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.811386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 418/852] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:20.815903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.849323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4022102-422c-4987-8255-52733213248d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:20.867424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:20.891534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:20.892543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:20.892932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:20.893435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:20.905859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:20.907531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:20.926047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:20.927102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:20.928198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:20.928198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:20.928198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:20.928198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:20.928800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 435/853] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:20.934372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:20.965112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa350ed-f8ad-4d66-acc9-d73b1b533e1a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.020417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.021066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.021346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.021618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.029729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.031228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:21.046261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:21.047282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:21.048238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 419/854] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:21.054968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:21.057552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8ace9-37ef-4030-8d52-8cb6a6393eb7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.075972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:21.105978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.106661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.106950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.107248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.107986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:21.108297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.109696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:21.134339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 436/855] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:21.135602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:21.167613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:21.177431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8438f73-fd43-41c5-ac16-47b7e004af82 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.213299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.213696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.213985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.214264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.215170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:21.215530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.216917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:21.239678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 420/856] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 08:08:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:21.240812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:21.260720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752996ef-ade7-48df-a9f8-0e5659a4ef02 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.302989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.303995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.304286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.304574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.313469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.315270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:21.332799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:21.333132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:21.333966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:21.334844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:21.334844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:21.334844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:21.334844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:21.334844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:21.334844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:21.334844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:21.334844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:21.339359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 437/857] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:21.340764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:21.378888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a11a74-569b-4b03-aae5-09dc9e184027 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.415677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6fa48-d64f-487e-a5a9-030e1f044c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.416437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6fa48-d64f-487e-a5a9-030e1f044c81 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.416891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6fa48-d64f-487e-a5a9-030e1f044c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.417066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6fa48-d64f-487e-a5a9-030e1f044c81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.445502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:21.459603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bddf5-c617-4881-b08e-282a76d8c4fb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.494022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6fa48-d64f-487e-a5a9-030e1f044c81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.495736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6fa48-d64f-487e-a5a9-030e1f044c81 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:21.500683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64958e0d-b169-4a46-a2a5-96b6ed229120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.501429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64958e0d-b169-4a46-a2a5-96b6ed229120 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.501772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64958e0d-b169-4a46-a2a5-96b6ed229120 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.502106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64958e0d-b169-4a46-a2a5-96b6ed229120 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.502962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64958e0d-b169-4a46-a2a5-96b6ed229120 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:21.503379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64958e0d-b169-4a46-a2a5-96b6ed229120 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.504922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64958e0d-b169-4a46-a2a5-96b6ed229120 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:21.517653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 438/858] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:21.520763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e8572-ebac-45ef-a60e-062eb47fddcf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.521041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e8572-ebac-45ef-a60e-062eb47fddcf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.521315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e8572-ebac-45ef-a60e-062eb47fddcf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.522591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 439/859] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:21.523299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e8572-ebac-45ef-a60e-062eb47fddcf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:21.576986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:21.611549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 421/860] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:08:21.617390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872aa37-f2b5-426e-bc82-40dc20d49718 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.617677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872aa37-f2b5-426e-bc82-40dc20d49718 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.617956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872aa37-f2b5-426e-bc82-40dc20d49718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.634561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9872aa37-f2b5-426e-bc82-40dc20d49718 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:21.635244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e8572-ebac-45ef-a60e-062eb47fddcf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.635786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9872aa37-f2b5-426e-bc82-40dc20d49718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:21.719236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 422/861] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:21.719236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872aa37-f2b5-426e-bc82-40dc20d49718 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:21.791426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.791995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.792271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.792540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.793373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=70d16599118b4968a18bf3f783a6687f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:21.793740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.795230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:21.811246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 440/862] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:21 2022] GET /identity/v3/role_assignments?user.id=70d16599118b4968a18bf3f783a6687f&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:21.811246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:21.825168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:21.846327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872aa37-f2b5-426e-bc82-40dc20d49718 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.848346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:21.887566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.888415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.888689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.888959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.902627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.904159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:21.923359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:21.924332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:21.925216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:21.925216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:21.925216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:21.925216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:21.925216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 423/863] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:21.928868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f0f54-1fde-4b1a-b219-1728973a3892 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:21.931818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:21.981984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636b3bb-5706-4887-9942-5f7642910834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:21.983656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636b3bb-5706-4887-9942-5f7642910834 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:21.984183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636b3bb-5706-4887-9942-5f7642910834 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:21.984674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636b3bb-5706-4887-9942-5f7642910834 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:21.985879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636b3bb-5706-4887-9942-5f7642910834 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:21.988507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636b3bb-5706-4887-9942-5f7642910834 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:22.007266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 441/864] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:08:21 2022] POST /identity/v3/services => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:22.055825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456317ea-c9de-48f9-935b-0f008d0cb5b2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:22.108640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:22.109736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:22.110231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:22.110695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:22.111732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:22.112258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:22.113985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:22.128703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 424/865] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:22.134306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636b3bb-5706-4887-9942-5f7642910834 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:22.215412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:22.249363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636b3bb-5706-4887-9942-5f7642910834 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:22.284779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:22.285644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:22.285925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:22.286199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:22.293771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:22.295296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:22.310166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:22.311065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:22.311982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 442/866] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:22.317097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:22.330497 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:22.423412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fd9fc-e0af-4a5c-8317-708c4af1ee50 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:22.461204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c227171-6898-48f1-941a-21e794465072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:22.462210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227171-6898-48f1-941a-21e794465072 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:22.462665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227171-6898-48f1-941a-21e794465072 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:22.463110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227171-6898-48f1-941a-21e794465072 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:22.464088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c227171-6898-48f1-941a-21e794465072 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:22.464589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c227171-6898-48f1-941a-21e794465072 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:22.466208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c227171-6898-48f1-941a-21e794465072 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:22.479689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 425/867] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:22.485853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:22.590248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b4d9f-85a3-4ba3-81ef-45c9142d1565 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:22.619393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:22.625620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078374b3-c1bd-4630-acdc-6bd986a2efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:22.626288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078374b3-c1bd-4630-acdc-6bd986a2efb3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:22.626563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078374b3-c1bd-4630-acdc-6bd986a2efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:22.626840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078374b3-c1bd-4630-acdc-6bd986a2efb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:22.627735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078374b3-c1bd-4630-acdc-6bd986a2efb3 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=7356a0f8d9f84e7cb785662db9afd658) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:22.628047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078374b3-c1bd-4630-acdc-6bd986a2efb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:22.629690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078374b3-c1bd-4630-acdc-6bd986a2efb3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:22.636376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 443/868] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:22 2022] GET /identity/v3/role_assignments?user.id=7356a0f8d9f84e7cb785662db9afd658&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:22.877794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb08c0-4c9c-4ecb-8c15-7dc9ed97f171 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:22.878200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb08c0-4c9c-4ecb-8c15-7dc9ed97f171 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:22.878394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb08c0-4c9c-4ecb-8c15-7dc9ed97f171 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:22.880027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 426/869] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:22.887829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45528-b9f2-4654-a27f-938bbe3b3439 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:22.888235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45528-b9f2-4654-a27f-938bbe3b3439 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:22.888622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45528-b9f2-4654-a27f-938bbe3b3439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:22.906943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c6d45528-b9f2-4654-a27f-938bbe3b3439 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:22.908171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c6d45528-b9f2-4654-a27f-938bbe3b3439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:22.982039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 444/870] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:23.004980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb08c0-4c9c-4ecb-8c15-7dc9ed97f171 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:23.080351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:23.115912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb08c0-4c9c-4ecb-8c15-7dc9ed97f171 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:23.152643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:23.153799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:23.154278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:23.154813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:23.155730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:23.157299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:23.170361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:23.171371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:23.172212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:23.175224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 427/871] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 23 08:08:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:23.183355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45528-b9f2-4654-a27f-938bbe3b3439 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:23.286369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45528-b9f2-4654-a27f-938bbe3b3439 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:23.312522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:23.322263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65558cd4-487e-4280-b47f-ba7e95f0386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:23.322941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558cd4-487e-4280-b47f-ba7e95f0386a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:23.323246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558cd4-487e-4280-b47f-ba7e95f0386a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:23.323512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558cd4-487e-4280-b47f-ba7e95f0386a None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:23.324215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65558cd4-487e-4280-b47f-ba7e95f0386a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:23.324519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65558cd4-487e-4280-b47f-ba7e95f0386a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:23.325898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65558cd4-487e-4280-b47f-ba7e95f0386a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:23.331048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 445/872] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:23.336851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:23.443151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ea16-20c8-4a92-ab05-b5c7f853a792 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:23.477817 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9dfeb-8496-4742-be88-022315b80d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:23.478839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9dfeb-8496-4742-be88-022315b80d4b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:23.479356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9dfeb-8496-4742-be88-022315b80d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:23.479888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9dfeb-8496-4742-be88-022315b80d4b None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:23.480831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9dfeb-8496-4742-be88-022315b80d4b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:23.481373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9dfeb-8496-4742-be88-022315b80d4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:23.482943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9dfeb-8496-4742-be88-022315b80d4b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:23.484369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:23.489198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 428/873] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:23 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:24.047804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67660ad2-b791-43d4-a761-6065eaba5abb None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.048845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67660ad2-b791-43d4-a761-6065eaba5abb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.049343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67660ad2-b791-43d4-a761-6065eaba5abb None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.051263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 446/874] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:24.056756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bfdcb-426c-4299-9b5e-a066d3f05b42 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.057287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bfdcb-426c-4299-9b5e-a066d3f05b42 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.057859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bfdcb-426c-4299-9b5e-a066d3f05b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.074903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3c4bfdcb-426c-4299-9b5e-a066d3f05b42 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:24.076415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3c4bfdcb-426c-4299-9b5e-a066d3f05b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:24.143645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 429/875] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:24.168473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67660ad2-b791-43d4-a761-6065eaba5abb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:24.200305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ae9b-c0ea-4993-b688-873fec165573 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.200928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ae9b-c0ea-4993-b688-873fec165573 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.201435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ae9b-c0ea-4993-b688-873fec165573 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.203429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 430/876] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:24.208947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973ebb7-470f-4a08-959a-c7b9b5d89512 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.209510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973ebb7-470f-4a08-959a-c7b9b5d89512 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.209989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973ebb7-470f-4a08-959a-c7b9b5d89512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.229184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c973ebb7-470f-4a08-959a-c7b9b5d89512 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:24.231249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c973ebb7-470f-4a08-959a-c7b9b5d89512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:24.282066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67660ad2-b791-43d4-a761-6065eaba5abb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:24.316703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b73ee-1f38-4b00-b472-a40c4f7dd30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:24.317532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b73ee-1f38-4b00-b472-a40c4f7dd30b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.317811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b73ee-1f38-4b00-b472-a40c4f7dd30b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.318079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b73ee-1f38-4b00-b472-a40c4f7dd30b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.323395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 431/877] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:24.332990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b73ee-1f38-4b00-b472-a40c4f7dd30b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-098b73ee-1f38-4b00-b472-a40c4f7dd30b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:24.341385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:24.342313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:24.343149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:24.345968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973ebb7-470f-4a08-959a-c7b9b5d89512 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:24.348286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 447/878] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:24.356187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b73ee-1f38-4b00-b472-a40c4f7dd30b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:24.415470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:24.417551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:24.465550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973ebb7-470f-4a08-959a-c7b9b5d89512 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:24.479485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b73ee-1f38-4b00-b472-a40c4f7dd30b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:24.511968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:24.513030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.513501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.514056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.514943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:24.516667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:24.529179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27032f60-1634-4898-9194-63cf1624bead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:24.529862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27032f60-1634-4898-9194-63cf1624bead None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.530147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27032f60-1634-4898-9194-63cf1624bead None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.530424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27032f60-1634-4898-9194-63cf1624bead None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:24.531014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:24.532015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:24.532843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:24.533558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27032f60-1634-4898-9194-63cf1624bead None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:24.533558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27032f60-1634-4898-9194-63cf1624bead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:24.533558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27032f60-1634-4898-9194-63cf1624bead None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:24.540974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 432/879] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:24.542115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:24.547397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 448/880] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:24.550108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27032f60-1634-4898-9194-63cf1624bead None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:24.656818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f0018-4a09-4f6b-a858-a3dce3bf87cc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:24.679640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27032f60-1634-4898-9194-63cf1624bead None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:24.687579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:24.695188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:24.704474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:24.705223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.706007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.706310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.707245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:24.707560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:24.709513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:24.720936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 433/881] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:24.721941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:24.726188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c987a6f-b186-4264-b9e7-55184cecfb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:24.727741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c987a6f-b186-4264-b9e7-55184cecfb9f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.728021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c987a6f-b186-4264-b9e7-55184cecfb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.728296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c987a6f-b186-4264-b9e7-55184cecfb9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.744656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c987a6f-b186-4264-b9e7-55184cecfb9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:24.746755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c987a6f-b186-4264-b9e7-55184cecfb9f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6c987a6f-b186-4264-b9e7-55184cecfb9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:24.768853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:24.769963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:24.770841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:24.770841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:24.770841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:24.770841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:24.771549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 449/882] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:24.772863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a0431-9168-44de-8c22-218c621a666a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.773145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a0431-9168-44de-8c22-218c621a666a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.773417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a0431-9168-44de-8c22-218c621a666a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.779245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 450/883] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:24.779992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a0431-9168-44de-8c22-218c621a666a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:24.831458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4489b-d242-46b5-8ec0-d8d734bbc8c2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:24.866950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d955ad6-8b6b-43b9-a322-2d7ad51633db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:24.867651 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d955ad6-8b6b-43b9-a322-2d7ad51633db None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.867978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d955ad6-8b6b-43b9-a322-2d7ad51633db None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.868215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d955ad6-8b6b-43b9-a322-2d7ad51633db None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.868922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d955ad6-8b6b-43b9-a322-2d7ad51633db None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:24.869264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d955ad6-8b6b-43b9-a322-2d7ad51633db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:24.870610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d955ad6-8b6b-43b9-a322-2d7ad51633db None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:24.876339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 434/884] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:24.877576 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4fee6-3daf-47c5-96b3-b8ae0a14acb8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.877873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4fee6-3daf-47c5-96b3-b8ae0a14acb8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.878150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4fee6-3daf-47c5-96b3-b8ae0a14acb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.894315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-25c4fee6-3daf-47c5-96b3-b8ae0a14acb8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:24.894657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a0431-9168-44de-8c22-218c621a666a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:24.895690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-25c4fee6-3daf-47c5-96b3-b8ae0a14acb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:24.930200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e0212-ff88-416c-ad78-146024399099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:24.930787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e0212-ff88-416c-ad78-146024399099 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:24.931069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e0212-ff88-416c-ad78-146024399099 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:24.931378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e0212-ff88-416c-ad78-146024399099 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:24.932124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e0212-ff88-416c-ad78-146024399099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:24.932419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e0212-ff88-416c-ad78-146024399099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:24.933814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e0212-ff88-416c-ad78-146024399099 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:24.944004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:24.950399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 451/885] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:24.950399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e0212-ff88-416c-ad78-146024399099 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:24.975551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:24.983547 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 435/886] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:24.984631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4fee6-3daf-47c5-96b3-b8ae0a14acb8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.057145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e0212-ff88-416c-ad78-146024399099 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.088896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4fee6-3daf-47c5-96b3-b8ae0a14acb8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.092265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.092932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.093212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.093486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.094275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] RBAC: Adding query filter params (interface=public, service_id=e27f56a5864041b3bcb2f8c113abc3cb, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:25.094586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.095995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.102495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 452/887] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/endpoints?service_id=e27f56a5864041b3bcb2f8c113abc3cb&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:25.104143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.125844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.126722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.127003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.127297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.134731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.136260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:25.151682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:25.153077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 436/888] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:08:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:25.153994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.200362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:25.203755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:25.218086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f7247-7d42-47d0-9eb8-b229017cb5b3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.253613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.254443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.254725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.254999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.255686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.257131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.262250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a5c94-8630-48eb-bf57-48f5d3325491 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:25.270605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:25.271632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:25.272468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 453/889] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:25.273269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.302079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec52ce-6d91-4f9f-b85a-878b95de0be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.303357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec52ce-6d91-4f9f-b85a-878b95de0be5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.303738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec52ce-6d91-4f9f-b85a-878b95de0be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.304110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec52ce-6d91-4f9f-b85a-878b95de0be5 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.304871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec52ce-6d91-4f9f-b85a-878b95de0be5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.306438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec52ce-6d91-4f9f-b85a-878b95de0be5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.312212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 437/890] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:08:25 2022] GET /identity/v3/services => generated 1011 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:08:25.313064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec52ce-6d91-4f9f-b85a-878b95de0be5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.426923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec52ce-6d91-4f9f-b85a-878b95de0be5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.430540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:25.432202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f64802-a1ca-4158-aa00-11a7040ec6b5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.473551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.474765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.475362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.475894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.476895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:25.477277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.478734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.479387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.480284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.480564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.480828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.481663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:25.481964 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.483582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.489603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 438/891] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:25.495672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.512576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 454/892] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 08:08:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:25.518066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.531029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:25.605268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bb26d-73b0-43d6-b560-6ea069b8e745 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.631681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07613-1ced-4eff-bda0-8b2adc2fa28c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.639559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc568a-f39f-41b2-bcde-fccc4e7fb7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.640229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc568a-f39f-41b2-bcde-fccc4e7fb7ac None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.640507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc568a-f39f-41b2-bcde-fccc4e7fb7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.640780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc568a-f39f-41b2-bcde-fccc4e7fb7ac None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.641491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc568a-f39f-41b2-bcde-fccc4e7fb7ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:25.641796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc568a-f39f-41b2-bcde-fccc4e7fb7ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.643159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc568a-f39f-41b2-bcde-fccc4e7fb7ac None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.654679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 439/893] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:25 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:25.654679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115319f-6e29-4713-8c1a-b69e62b38bce None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.654679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115319f-6e29-4713-8c1a-b69e62b38bce None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.654679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115319f-6e29-4713-8c1a-b69e62b38bce None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.654679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 440/894] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:25.656569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4214d8-04cd-4a38-8738-aab8fcc46c88 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.656858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4214d8-04cd-4a38-8738-aab8fcc46c88 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.657123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4214d8-04cd-4a38-8738-aab8fcc46c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.674625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-dc4214d8-04cd-4a38-8738-aab8fcc46c88 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:25.675953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dc4214d8-04cd-4a38-8738-aab8fcc46c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:25.677275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f59f1-cb8e-4ac3-840a-3a970297f9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.678243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f59f1-cb8e-4ac3-840a-3a970297f9fb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.678613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f59f1-cb8e-4ac3-840a-3a970297f9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.678957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f59f1-cb8e-4ac3-840a-3a970297f9fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.706056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:25.756484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f59f1-cb8e-4ac3-840a-3a970297f9fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.756891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 441/895] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:25 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:25.758306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f59f1-cb8e-4ac3-840a-3a970297f9fb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.772303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:25.775626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4214d8-04cd-4a38-8738-aab8fcc46c88 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.888878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 455/896] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:25 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 08:08:25.900640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f59f1-cb8e-4ac3-840a-3a970297f9fb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:25.914714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4214d8-04cd-4a38-8738-aab8fcc46c88 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:25.953439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e2fb4-7396-4460-aea5-b2babdde78d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:25.954191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e2fb4-7396-4460-aea5-b2babdde78d6 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:25.954477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e2fb4-7396-4460-aea5-b2babdde78d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:25.954754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e2fb4-7396-4460-aea5-b2babdde78d6 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:25.955550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e2fb4-7396-4460-aea5-b2babdde78d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:25.957293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e2fb4-7396-4460-aea5-b2babdde78d6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:25.982276 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 442/897] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:08:25 2022] POST /identity/v3/services => generated 269 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:26.029534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f59f1-cb8e-4ac3-840a-3a970297f9fb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:26.043029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:26.078223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:26.079099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:26.079404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:26.079675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:26.091734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:26.093293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:26.114086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:26.115042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:26.116050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:26.116050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:26.116050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:26.116050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:26.116050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 456/898] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:26.121684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e2fb4-7396-4460-aea5-b2babdde78d6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:26.150244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:26.236544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e2fb4-7396-4460-aea5-b2babdde78d6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:26.273262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:26.273936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:26.274220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:26.274496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:26.275254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:26.275569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:26.276952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:26.290368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 443/899] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:26.295692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:26.401039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8f209-74a2-4b29-aef8-f2cc8ab8b03c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:26.436732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:26.437541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:26.437821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:26.438100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:26.445727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:26.447256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:26.462596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:26.463575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:26.464452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 457/900] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:26.469876 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:26.561467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:26.579979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a21a5f-a055-4909-b259-9989f918f2b6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:26.615011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bfebd-40c6-41fa-a2f4-8f492e40d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:26.615703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bfebd-40c6-41fa-a2f4-8f492e40d52a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:26.615985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bfebd-40c6-41fa-a2f4-8f492e40d52a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:26.616260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bfebd-40c6-41fa-a2f4-8f492e40d52a None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:26.617045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bfebd-40c6-41fa-a2f4-8f492e40d52a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:26.617353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bfebd-40c6-41fa-a2f4-8f492e40d52a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:26.618785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bfebd-40c6-41fa-a2f4-8f492e40d52a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:26.631102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 444/901] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:26.637074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:26.657308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:26.741416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea141d53-2312-45f4-9140-3370aad36bec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:26.776403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d1944-4cd9-4d0a-807b-f434ec3c985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:26.777066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1944-4cd9-4d0a-807b-f434ec3c985d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:26.777348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1944-4cd9-4d0a-807b-f434ec3c985d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:26.777623 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1944-4cd9-4d0a-807b-f434ec3c985d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:26.778452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d1944-4cd9-4d0a-807b-f434ec3c985d None admin] RBAC: Adding query filter params (role.id=52fd1ff24fe24ad98ca611cd5f52669f, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=70d16599118b4968a18bf3f783a6687f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:26.778764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d1944-4cd9-4d0a-807b-f434ec3c985d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:26.780225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d1944-4cd9-4d0a-807b-f434ec3c985d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:26.786703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 458/902] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:26 2022] GET /identity/v3/role_assignments?user.id=70d16599118b4968a18bf3f783a6687f&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=52fd1ff24fe24ad98ca611cd5f52669f => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:27.385145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375dc8-61ce-4d82-a7d9-46c8e2cc14aa None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:27.385682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375dc8-61ce-4d82-a7d9-46c8e2cc14aa None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:27.385682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375dc8-61ce-4d82-a7d9-46c8e2cc14aa None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:27.387306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 445/903] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:27.392699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ac8b7-f776-410d-8a6a-c814db0adb38 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:27.393095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ac8b7-f776-410d-8a6a-c814db0adb38 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:27.393257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ac8b7-f776-410d-8a6a-c814db0adb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:27.410475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b64ac8b7-f776-410d-8a6a-c814db0adb38 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:27.411873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b64ac8b7-f776-410d-8a6a-c814db0adb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:27.482175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 459/904] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:27.505509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71375dc8-61ce-4d82-a7d9-46c8e2cc14aa None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:27.613261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71375dc8-61ce-4d82-a7d9-46c8e2cc14aa None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:27.649677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:27.650402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:27.650684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:27.650960 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:27.651649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:27.653031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:27.666487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:27.668675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:27.670584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 446/905] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:27.673745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ac8b7-f776-410d-8a6a-c814db0adb38 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:27.692787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:27.778572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ac8b7-f776-410d-8a6a-c814db0adb38 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:27.817513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a837c5-63fd-4b82-b653-49e716e66fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:27.818195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a837c5-63fd-4b82-b653-49e716e66fac None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:27.818479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a837c5-63fd-4b82-b653-49e716e66fac None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:27.818755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a837c5-63fd-4b82-b653-49e716e66fac None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:27.819493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a837c5-63fd-4b82-b653-49e716e66fac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:27.819804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a837c5-63fd-4b82-b653-49e716e66fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:27.821192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a837c5-63fd-4b82-b653-49e716e66fac None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:27.826289 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 460/906] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:27.832001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:27.841991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:27.875174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba805-1ba7-487f-b9d9-7a4f644decc7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:27.875511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba805-1ba7-487f-b9d9-7a4f644decc7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:27.875768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba805-1ba7-487f-b9d9-7a4f644decc7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:27.878727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 461/907] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:27.883403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490de3-8c46-4447-a5ff-cd1fc8b3df49 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:27.883714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490de3-8c46-4447-a5ff-cd1fc8b3df49 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:27.884005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490de3-8c46-4447-a5ff-cd1fc8b3df49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:27.901299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4c490de3-8c46-4447-a5ff-cd1fc8b3df49 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:27.902416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4c490de3-8c46-4447-a5ff-cd1fc8b3df49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:27.945745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f462c-7fce-4013-b0ef-eb11dcf5667a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:27.980412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 462/908] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:27.982254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:27.983176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:27.983718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:27.984202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:27.985113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:27.985620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:27.987188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:27.993080 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 447/909] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 08:08:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:27.999328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c490de3-8c46-4447-a5ff-cd1fc8b3df49 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:28.000686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:28.008725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:28.073593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:28.106754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c490de3-8c46-4447-a5ff-cd1fc8b3df49 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:28.107975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6c357-9ee5-4a28-bb81-af7775366a07 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:28.143000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79ca8-d393-4ebe-802a-8a35cddebc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:28.143346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bdb55-5217-4f4a-bee4-c250c0670e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:28.144006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bdb55-5217-4f4a-bee4-c250c0670e0c None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.144347 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bdb55-5217-4f4a-bee4-c250c0670e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.144616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bdb55-5217-4f4a-bee4-c250c0670e0c None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.145006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79ca8-d393-4ebe-802a-8a35cddebc4f None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.145349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bdb55-5217-4f4a-bee4-c250c0670e0c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:28.145692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79ca8-d393-4ebe-802a-8a35cddebc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.146160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79ca8-d393-4ebe-802a-8a35cddebc4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.146722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bdb55-5217-4f4a-bee4-c250c0670e0c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:28.147428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79ca8-d393-4ebe-802a-8a35cddebc4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:28.149016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79ca8-d393-4ebe-802a-8a35cddebc4f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:28.161303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 448/910] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:08:27 2022] POST /identity/v3/services => generated 241 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:28.162650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8eafb-d2e1-47ea-baf3-b5a6d933c42a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.162938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8eafb-d2e1-47ea-baf3-b5a6d933c42a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.163232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8eafb-d2e1-47ea-baf3-b5a6d933c42a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.164626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 449/911] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:28.170307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdc85d-f107-4333-bb96-825fec9efede None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.170595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdc85d-f107-4333-bb96-825fec9efede None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.170875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdc85d-f107-4333-bb96-825fec9efede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.184045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 463/912] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:08:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:28.193702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-eacdc85d-f107-4333-bb96-825fec9efede None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:28.195392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-eacdc85d-f107-4333-bb96-825fec9efede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:28.296042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 450/913] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:28.321779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79ca8-d393-4ebe-802a-8a35cddebc4f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:28.395078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:28.439612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79ca8-d393-4ebe-802a-8a35cddebc4f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:28.475886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:28.476719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.477002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.477279 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.477942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:28.479361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:28.492189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:28.493165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:28.493981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 464/914] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 23 08:08:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:28.499856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdc85d-f107-4333-bb96-825fec9efede None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:28.564608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:28.616446 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdc85d-f107-4333-bb96-825fec9efede None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:28.651761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:28.652546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.652827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.653098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.653806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:28.654113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:28.655506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:28.660811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 451/915] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:28.666435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:28.771934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c9bc4-8b28-4050-b793-95edc6d22781 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:28.798708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:28.808770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108c6bb-05f2-47dd-a2b5-41da341246b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:28.809724 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108c6bb-05f2-47dd-a2b5-41da341246b3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.810266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108c6bb-05f2-47dd-a2b5-41da341246b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.810754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108c6bb-05f2-47dd-a2b5-41da341246b3 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.811728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108c6bb-05f2-47dd-a2b5-41da341246b3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:28.812226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108c6bb-05f2-47dd-a2b5-41da341246b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:28.813796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108c6bb-05f2-47dd-a2b5-41da341246b3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:28.823328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 465/916] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:28.828275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:28.891788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad01d5-f878-4192-8cc1-644ba5b2abbc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.892113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad01d5-f878-4192-8cc1-644ba5b2abbc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.892340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad01d5-f878-4192-8cc1-644ba5b2abbc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.893963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 466/917] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:28.899031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b1651-d61b-4a27-a1c6-9c30296ac33b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.899363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b1651-d61b-4a27-a1c6-9c30296ac33b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.899647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b1651-d61b-4a27-a1c6-9c30296ac33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.920640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-131b1651-d61b-4a27-a1c6-9c30296ac33b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:28.922184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-131b1651-d61b-4a27-a1c6-9c30296ac33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:28.936357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849b71f-b45a-430a-9dce-9d74139759cf None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:28.959177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:28.975457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:28.976172 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:28.976448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:28.976716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:28.977550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] RBAC: Adding query filter params (interface=public, service_id=764d2653a86141fa916634e985209534, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:28.977860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:28.979263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:28.989767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 452/918] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 08:08:28 2022] GET /identity/v3/endpoints?service_id=764d2653a86141fa916634e985209534&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:28.992379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:29.014605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 467/919] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:29.037367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b1651-d61b-4a27-a1c6-9c30296ac33b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:29.112128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a435e6a-922e-48e1-bcd0-48c9e7c4ff2d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:29.143349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b1651-d61b-4a27-a1c6-9c30296ac33b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:29.151175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be79d59-1e42-417c-a6b4-5c8b43347f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:29.152253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79d59-1e42-417c-a6b4-5c8b43347f38 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:29.152534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79d59-1e42-417c-a6b4-5c8b43347f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:29.152810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79d59-1e42-417c-a6b4-5c8b43347f38 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:29.153828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be79d59-1e42-417c-a6b4-5c8b43347f38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:29.155572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be79d59-1e42-417c-a6b4-5c8b43347f38 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:29.165485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:29.167434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 453/920] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:08:28 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:08:29.187306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8790-fa28-4454-b429-b1dd54254d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:29.188453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8790-fa28-4454-b429-b1dd54254d95 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:29.188860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8790-fa28-4454-b429-b1dd54254d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:29.189286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8790-fa28-4454-b429-b1dd54254d95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:29.202041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba8790-fa28-4454-b429-b1dd54254d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fbba8790-fa28-4454-b429-b1dd54254d95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:29.210730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:29.211810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:29.212644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 468/921] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 08:08:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:29.220588 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be79d59-1e42-417c-a6b4-5c8b43347f38 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:29.283918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:29.345670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be79d59-1e42-417c-a6b4-5c8b43347f38 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:29.386703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:29.387416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:29.387703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:29.387987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:29.388741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:29.389049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:29.390429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:29.399868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 454/922] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 08:08:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:29.405178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8790-fa28-4454-b429-b1dd54254d95 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:29.529706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8790-fa28-4454-b429-b1dd54254d95 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:29.573456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:29.574651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:29.575153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:29.575664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:29.587336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:29.589139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:29.606688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:29.607687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:29.608723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:29.608723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:29.608723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:29.608723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:29.611034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 469/923] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:29.616710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:29.722037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9309a02-8a35-4fe3-b24e-df5caca302ef None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:29.732766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:29.757259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:29.757932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:29.758216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:29.758492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:29.759236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:29.759563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:29.760944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:29.773693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 455/924] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:29.778993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:29.811898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:29.880451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b027d2-2e23-411f-9752-5f2c4586f975 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:29.925461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:29.926837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:29.927123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:29.927442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:29.935389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:29.936868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:29.952951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:29.953978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:29.955304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 470/925] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:08:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:29.960415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.066186 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcef2-953d-4a3b-96a2-980de9d6196a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:30.101090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:30.101780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.102063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.102343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.103188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:30.103534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:30.104981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:30.123292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 456/926] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 08:08:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:30.129247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.221738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:30.233345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d1ed-316f-4f66-9aef-291f11a8b03b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:30.276177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7667e-fc29-466e-aed3-45dfcce7c5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:30.276996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7667e-fc29-466e-aed3-45dfcce7c5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.277276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7667e-fc29-466e-aed3-45dfcce7c5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.277548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7667e-fc29-466e-aed3-45dfcce7c5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.289027 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7667e-fc29-466e-aed3-45dfcce7c5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:30.290506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7667e-fc29-466e-aed3-45dfcce7c5f8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6ee7667e-fc29-466e-aed3-45dfcce7c5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:30.308738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:30.309675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:30.310560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:30.310560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:30.310560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:30.310560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:30.310560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 471/927] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:30.316739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.347814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:30.390669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e58b1-9112-419a-b81a-3ff1251d26a1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.391360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e58b1-9112-419a-b81a-3ff1251d26a1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.391841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e58b1-9112-419a-b81a-3ff1251d26a1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.393690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 472/928] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:30.400470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237247f-a5c4-4251-aca6-cee563160a02 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.400470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237247f-a5c4-4251-aca6-cee563160a02 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.400470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237247f-a5c4-4251-aca6-cee563160a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.418318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b237247f-a5c4-4251-aca6-cee563160a02 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:30.419594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b237247f-a5c4-4251-aca6-cee563160a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:30.433354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e8a01-dbd6-4ee5-999d-ae4dbfbd6d39 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:30.469334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc41309-76d1-4bcf-89b5-1abf191df20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:30.470000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41309-76d1-4bcf-89b5-1abf191df20f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.470397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41309-76d1-4bcf-89b5-1abf191df20f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.470678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41309-76d1-4bcf-89b5-1abf191df20f None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.471436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc41309-76d1-4bcf-89b5-1abf191df20f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:30.471799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc41309-76d1-4bcf-89b5-1abf191df20f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:30.473777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc41309-76d1-4bcf-89b5-1abf191df20f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:30.494226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 457/929] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:30.494226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7641ae-9541-4591-a40e-78c006a7e6ce None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.494226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7641ae-9541-4591-a40e-78c006a7e6ce None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.494226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7641ae-9541-4591-a40e-78c006a7e6ce None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.495258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 458/930] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:30.499535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7641ae-9541-4591-a40e-78c006a7e6ce None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.506534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 473/931] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:30.506534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37400dd-9fd1-4f5e-bb59-d19e8c47adcb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.506534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37400dd-9fd1-4f5e-bb59-d19e8c47adcb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.506534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37400dd-9fd1-4f5e-bb59-d19e8c47adcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.530618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b37400dd-9fd1-4f5e-bb59-d19e8c47adcb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:30.532129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b37400dd-9fd1-4f5e-bb59-d19e8c47adcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:30.601537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:30.607642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 474/932] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:30.608870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37400dd-9fd1-4f5e-bb59-d19e8c47adcb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.612908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:30.626383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7641ae-9541-4591-a40e-78c006a7e6ce None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:30.664940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:30.665739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.666025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.666302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.673332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:30.674777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:30.689718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:30.690649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 459/933] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:30.692295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.720383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37400dd-9fd1-4f5e-bb59-d19e8c47adcb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:30.758451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:30.759049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.759375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.759655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.760360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:30.761844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:30.763276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING keystone.server.flask.common [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 08:08:30.782151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 08:08:30.782151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 08:08:30.782151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [parameters: {'id': 'addfc46baaf543f4ad6d0a2880aedb23', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 08:08:30.782151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id addfc46baaf543f4ad6d0a2880aedb23.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id addfc46baaf543f4ad6d0a2880aedb23. Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 08:08:30.791454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 08:08:30.792369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() Jun 23 08:08:30.793338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 08:08:30.795515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 08:08:30.796441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 08:08:30.797226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'id': 'addfc46baaf543f4ad6d0a2880aedb23', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:30.799999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:30.802509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id addfc46baaf543f4ad6d0a2880aedb23. Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 475/934] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Jun 23 08:08:30 2022] POST /identity/v3/projects => generated 262 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:30.805494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.808418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948397f-97ef-4119-a070-7249d5c34314 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:30.850024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:30.850844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.851124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.851425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.852096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:30.853468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:30.859337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:30.867568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:30.868626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:30.869493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 460/935] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:30.870336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.887891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:30.912794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf877c-c9f1-4df4-9ed2-b925fceaf97a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:30.948227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:30.948978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:30.949331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:30.949658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:30.950504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:30.950869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:30.952618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:30.968036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 476/936] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:30.968999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:30.985228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332186a-de91-4b3e-a312-3cf11fea3ba9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.021304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.022109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.022448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.022868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.030292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.032076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:31.047179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:31.048304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:31.049280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 461/937] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:31.049793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:31.074201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416b1a7-3274-45df-9fe3-6472247d4da7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.095560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:31.110310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.110976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.111280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.111554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.112259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:31.112558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.114218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.124063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 477/938] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:31.124063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:31.129713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:31.162937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f599a-61a8-457d-83f1-d2751bd6af51 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.203312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2a416-4b46-49e8-8c9b-59d8af0af25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.204374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a416-4b46-49e8-8c9b-59d8af0af25c None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.204668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a416-4b46-49e8-8c9b-59d8af0af25c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.204921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a416-4b46-49e8-8c9b-59d8af0af25c None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/70d16599118b4968a18bf3f783a6687f/roles/52fd1ff24fe24ad98ca611cd5f52669f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.207233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2a416-4b46-49e8-8c9b-59d8af0af25c None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=70d16599118b4968a18bf3f783a6687f, role_id=52fd1ff24fe24ad98ca611cd5f52669f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.208656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2a416-4b46-49e8-8c9b-59d8af0af25c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.225452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d91f5-5d17-40aa-a79a-c13b2d1a3c3b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.231587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 462/939] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:08:31 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/70d16599118b4968a18bf3f783a6687f/roles/52fd1ff24fe24ad98ca611cd5f52669f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:08:31.232368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2a416-4b46-49e8-8c9b-59d8af0af25c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:31.272150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8a044-3aeb-4eed-9563-97e3ef1f9a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.272741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8a044-3aeb-4eed-9563-97e3ef1f9a31 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.273020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8a044-3aeb-4eed-9563-97e3ef1f9a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.273288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8a044-3aeb-4eed-9563-97e3ef1f9a31 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.274831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8a044-3aeb-4eed-9563-97e3ef1f9a31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:31.275143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8a044-3aeb-4eed-9563-97e3ef1f9a31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.277014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8a044-3aeb-4eed-9563-97e3ef1f9a31 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.304122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 478/940] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:31.344960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01eb10-cb9d-4b14-b02d-608214f1b57f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.345280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01eb10-cb9d-4b14-b02d-608214f1b57f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.345523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01eb10-cb9d-4b14-b02d-608214f1b57f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.347324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 479/941] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:31.353209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613049b8-3163-401f-8f93-bb1ae666e7c5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.353531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613049b8-3163-401f-8f93-bb1ae666e7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.353788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613049b8-3163-401f-8f93-bb1ae666e7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.378158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-613049b8-3163-401f-8f93-bb1ae666e7c5 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:31.380015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-613049b8-3163-401f-8f93-bb1ae666e7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:31.380275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2a416-4b46-49e8-8c9b-59d8af0af25c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.420200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:31.428040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.428908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.429262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.429731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.430646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:31.431896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.433321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.437341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:31.439944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 463/942] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:31.447298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:31.473324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 480/943] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:31.497898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613049b8-3163-401f-8f93-bb1ae666e7c5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:31.580857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0904e-a8de-46b8-b0a6-70f7b4610ef2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.623455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613049b8-3163-401f-8f93-bb1ae666e7c5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.626721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.627433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.627718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.627993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.629372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] RBAC: Adding query filter params (interface=public, service_id=4e64f8d6ec9148669f40818fe78e97e1, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:31.629688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.631075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.638838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 464/944] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 08:08:31 2022] GET /identity/v3/endpoints?service_id=4e64f8d6ec9148669f40818fe78e97e1&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:31.644049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:31.666569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.667799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.668501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.668894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.669625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.671129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:31.688949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:31.690114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 08:08:31.691089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:31.692090 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 481/945] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 23 08:08:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:31.699445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:31.716983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:31.756998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:31.762976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb990ba-1c77-4c4e-b33b-37b6b4352bd6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.801318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018aacc-5e5c-443d-916b-a52c1e0938e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.801941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018aacc-5e5c-443d-916b-a52c1e0938e4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.802230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018aacc-5e5c-443d-916b-a52c1e0938e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.802506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018aacc-5e5c-443d-916b-a52c1e0938e4 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.803193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018aacc-5e5c-443d-916b-a52c1e0938e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.804666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018aacc-5e5c-443d-916b-a52c1e0938e4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.811425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 465/946] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:08:31 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:08:31.841583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cf81d-e6b2-4bea-b174-14b58da5ec0a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:31.880477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:31.881476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:31.881918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:31.882333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:31.883197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:31.883715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:31.885287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:31.891506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 482/947] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:31.900409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018aacc-5e5c-443d-916b-a52c1e0938e4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:32.010125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018aacc-5e5c-443d-916b-a52c1e0938e4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:32.035029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:32.046490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d4f92-4367-411e-98ef-427427cfcbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:32.047165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4f92-4367-411e-98ef-427427cfcbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:32.047474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4f92-4367-411e-98ef-427427cfcbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:32.047752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4f92-4367-411e-98ef-427427cfcbf4 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:32.048469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d4f92-4367-411e-98ef-427427cfcbf4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:32.048779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d4f92-4367-411e-98ef-427427cfcbf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:32.050154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d4f92-4367-411e-98ef-427427cfcbf4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:32.055779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 466/948] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:32.061276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:32.161772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb31a16-f741-4f3d-9364-2fa6791ab0d6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:32.163397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:32.197809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd0574-ac11-48cc-babf-829d6fd67086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:32.198474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd0574-ac11-48cc-babf-829d6fd67086 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:32.198753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd0574-ac11-48cc-babf-829d6fd67086 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:32.199024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd0574-ac11-48cc-babf-829d6fd67086 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:32.199881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd0574-ac11-48cc-babf-829d6fd67086 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:32.201315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd0574-ac11-48cc-babf-829d6fd67086 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:32.224069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 483/949] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:08:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:33.399762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099beb88-849b-41bd-98c2-30ba105f3a17 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.400336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099beb88-849b-41bd-98c2-30ba105f3a17 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.400336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099beb88-849b-41bd-98c2-30ba105f3a17 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.401917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 467/950] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:33.407104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdd993-56c6-4afe-b0b9-3d4b5a5bea0b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.407545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdd993-56c6-4afe-b0b9-3d4b5a5bea0b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.407738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdd993-56c6-4afe-b0b9-3d4b5a5bea0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.424836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-40bdd993-56c6-4afe-b0b9-3d4b5a5bea0b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:33.426854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-40bdd993-56c6-4afe-b0b9-3d4b5a5bea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:33.504331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 484/951] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:33.519438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d9ed-d1ef-444f-b3af-eff17158ea27 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.519809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d9ed-d1ef-444f-b3af-eff17158ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.520176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d9ed-d1ef-444f-b3af-eff17158ea27 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.521891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 468/952] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:33.527075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4182a9-368b-4a19-8cae-376f9df36b52 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.527471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4182a9-368b-4a19-8cae-376f9df36b52 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.527842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4182a9-368b-4a19-8cae-376f9df36b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.530648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d9ed-d1ef-444f-b3af-eff17158ea27 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:33.545235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ea4182a9-368b-4a19-8cae-376f9df36b52 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:33.546644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ea4182a9-368b-4a19-8cae-376f9df36b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:33.635380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 485/953] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:33.650820 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d9ed-d1ef-444f-b3af-eff17158ea27 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:33.661010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4182a9-368b-4a19-8cae-376f9df36b52 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:33.678978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:33.697363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e284b44-1303-4b87-a342-7a4584b19821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:33.698190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284b44-1303-4b87-a342-7a4584b19821 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.698474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284b44-1303-4b87-a342-7a4584b19821 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.698750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284b44-1303-4b87-a342-7a4584b19821 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.712695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e284b44-1303-4b87-a342-7a4584b19821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9e284b44-1303-4b87-a342-7a4584b19821 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:33.724068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:33.725061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:33.725910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:33.733904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 469/954] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 08:08:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:33.735538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e284b44-1303-4b87-a342-7a4584b19821 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:33.785681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4182a9-368b-4a19-8cae-376f9df36b52 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:33.799342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:33.825554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:33.827540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.828039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.828545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.829427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:33.831061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:33.849377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 08:08:33.849377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 08:08:33.849377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [parameters: {'user_id': 'a9bd25b369144e878172730dc0841cba', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 08:08:33.849377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jun 23 08:08:33.860178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 08:08:33.861106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 08:08:33.862055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 08:08:33.863013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 08:08:33.863970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 08:08:33.864829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a9bd25b369144e878172730dc0841cba', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:33.865656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 08:08:33.866447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 08:08:33.869887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 08:08:33.869887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 08:08:33.869887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 08:08:33.869887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 08:08:33.869887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 08:08:33.869887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:33.871898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e284b44-1303-4b87-a342-7a4584b19821 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:33.872046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 486/955] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 08:08:33 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:33.875650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:33.910812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84d2d8-a41b-4422-8212-e088f1aede63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:33.911675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d2d8-a41b-4422-8212-e088f1aede63 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.911969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d2d8-a41b-4422-8212-e088f1aede63 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.912267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d2d8-a41b-4422-8212-e088f1aede63 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.913549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84d2d8-a41b-4422-8212-e088f1aede63 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:33.913997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84d2d8-a41b-4422-8212-e088f1aede63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:33.916064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84d2d8-a41b-4422-8212-e088f1aede63 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:33.928178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 470/956] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 08:08:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:33.931237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c0e4a-91a7-4d08-a6be-6b21eacc64f1 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.931237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c0e4a-91a7-4d08-a6be-6b21eacc64f1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.931237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c0e4a-91a7-4d08-a6be-6b21eacc64f1 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.932272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 471/957] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:33.937124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe955a6-d8a7-4ee6-90e6-69065d9204fb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:33.937483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe955a6-d8a7-4ee6-90e6-69065d9204fb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:33.937832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe955a6-d8a7-4ee6-90e6-69065d9204fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:33.958500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5fe955a6-d8a7-4ee6-90e6-69065d9204fb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:33.959755 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5fe955a6-d8a7-4ee6-90e6-69065d9204fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:33.968224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:33.996873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c994f-1ba0-4c2b-803a-1b2eea92cb9d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.032871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.033707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.033991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.034269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.039298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 472/958] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:34.039298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe955a6-d8a7-4ee6-90e6-69065d9204fb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.046597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.048480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:34.064583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:34.065748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 08:08:34.066949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:34.069033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 487/959] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:34.070491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.088440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:34.151525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe955a6-d8a7-4ee6-90e6-69065d9204fb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.176532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19a691-f185-472d-a2ea-f6bff1e7c4d6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.186973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.188024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.188433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.188865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.200359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.201909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.214405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.215256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.215544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.215821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.216481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.217874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.220593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:34.221656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:34.223017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:34.224264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:34.224264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:34.224264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:34.224264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:34.227936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 473/960] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:34.228782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:34.234348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:34.235914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:34.237569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 488/961] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:34.239505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.330024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:34.339551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577444e0-c424-4f40-a3aa-5bb8673a76ee None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.343993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be50baf-a98a-40f5-a230-a80c4a41354a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.373198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91a28c-4422-49f9-8f8d-767213298b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.373859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a28c-4422-49f9-8f8d-767213298b44 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.374135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a28c-4422-49f9-8f8d-767213298b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.374407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a28c-4422-49f9-8f8d-767213298b44 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.375230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91a28c-4422-49f9-8f8d-767213298b44 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:34.375547 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91a28c-4422-49f9-8f8d-767213298b44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.376963 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91a28c-4422-49f9-8f8d-767213298b44 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.378417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.379156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.379461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.379725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.380435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:34.380731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.382074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.396779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 489/962] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:34.397592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.413712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 474/963] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:34.415115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980366c-280f-45ec-ad73-930f52f0642e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.415448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980366c-280f-45ec-ad73-930f52f0642e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.415720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980366c-280f-45ec-ad73-930f52f0642e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.416992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 475/964] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:34.417740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980366c-280f-45ec-ad73-930f52f0642e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.469804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:34.525170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27303227-64cc-42bc-a2a0-313d61945712 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.535962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980366c-280f-45ec-ad73-930f52f0642e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.563248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dd167-863b-41e7-a99a-3bdf4a0894d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.563929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd167-863b-41e7-a99a-3bdf4a0894d1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.564209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd167-863b-41e7-a99a-3bdf4a0894d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.564484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd167-863b-41e7-a99a-3bdf4a0894d1 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.565202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dd167-863b-41e7-a99a-3bdf4a0894d1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:34.565510 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dd167-863b-41e7-a99a-3bdf4a0894d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.567070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dd167-863b-41e7-a99a-3bdf4a0894d1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.572504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 490/965] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:34.574153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a1380-5beb-4024-8c0d-49120e97d0a7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.574617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a1380-5beb-4024-8c0d-49120e97d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.575065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a1380-5beb-4024-8c0d-49120e97d0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.575424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.576510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.577022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.577496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.586564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.588320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.592704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-880a1380-5beb-4024-8c0d-49120e97d0a7 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:34.594215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-880a1380-5beb-4024-8c0d-49120e97d0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:34.604123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:34.605119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:34.606289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 476/966] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:34.607079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.622290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:34.673574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 491/967] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:34.674421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a1380-5beb-4024-8c0d-49120e97d0a7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.718353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ea81-c1d0-4cfb-a90e-aa4b1ea5d6a6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.729621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:34.751874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.752532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.752810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.753081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.753824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:34.754126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.755523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.760354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 477/968] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:34.761134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.776660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a1380-5beb-4024-8c0d-49120e97d0a7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.810515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.811183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.811486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.811759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.812583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:34.812885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.814435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.831897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 492/969] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:34.832695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.838327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:34.865010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb22df-a6fa-4d80-93f8-4d640d31cd13 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.900430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.901227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.901503 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.901772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.902423 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.903815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:34.916033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:34.917205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:34.918109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 478/970] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:34.918895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:34.934098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9c914-600a-4645-8542-44bd6638a05a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:34.947682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:34.969665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedb8da-e128-4b52-9bc4-573c764700be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:34.970320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedb8da-e128-4b52-9bc4-573c764700be None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:34.970603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedb8da-e128-4b52-9bc4-573c764700be None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:34.970870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedb8da-e128-4b52-9bc4-573c764700be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:34.971632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedb8da-e128-4b52-9bc4-573c764700be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:34.973032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedb8da-e128-4b52-9bc4-573c764700be None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:34.991544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 493/971] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:08:34 2022] POST /identity/v3/endpoints => generated 349 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:34.992276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedb8da-e128-4b52-9bc4-573c764700be None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:35.025980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcec23f-0510-4fbf-979e-038a6c55fe84 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:35.070140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17667fc1-2335-4184-afc5-e0bfee2798d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:35.071126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17667fc1-2335-4184-afc5-e0bfee2798d1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:35.071444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17667fc1-2335-4184-afc5-e0bfee2798d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:35.071725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17667fc1-2335-4184-afc5-e0bfee2798d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:35.074670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:35.110609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedb8da-e128-4b52-9bc4-573c764700be None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:35.148581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:35.148581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:35.148897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:35.149132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:35.149877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] RBAC: Adding query filter params (name=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:35.150187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:35.150318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17667fc1-2335-4184-afc5-e0bfee2798d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:35.151601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:35.151943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17667fc1-2335-4184-afc5-e0bfee2798d1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:35.157054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 494/972] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:35.163096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:35.201866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:35.264664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 479/973] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:34 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:08:35.270139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17667fc1-2335-4184-afc5-e0bfee2798d1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:35.275272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61b6cc-ff32-4251-826d-6d323e13f2fa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:35.306688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:35.311503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:35.312180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:35.312461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:35.312733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:35.313443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:35.313748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:35.315168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:35.320330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 495/974] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:35.387694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17667fc1-2335-4184-afc5-e0bfee2798d1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:35.429337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:35.430184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:35.430439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:35.430713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:35.443177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:35.444688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:35.464777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:35.465733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:35.466624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:35.466624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:35.466624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:35.466624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:35.466624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 480/975] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:35.472237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:35.572673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:35.579685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f1dd-6674-40e8-8c74-ae4af3e518d4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:35.615150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:35.615853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:35.616108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:35.616415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:35.617139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:35.617444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:35.618850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:35.631836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 496/976] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:35.637441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:35.726004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:35.741607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc116b0-78a5-4232-b3fc-1899c83d37ff None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:35.777774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:35.778577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:35.778857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:35.779124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:35.786072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:35.787543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:35.801496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:35.802404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:35.803404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:35.803404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:35.803404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:35.803404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:35.803404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:35.803404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:35.803404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:35.803404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:35.807391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 481/977] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:35.808641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:35.913351 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea429ce2-29fc-4082-83d9-7dd1240a8d79 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:35.948110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e6245-6275-46c2-925a-f6db8a0114a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:35.948777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6245-6275-46c2-925a-f6db8a0114a1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:35.949036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6245-6275-46c2-925a-f6db8a0114a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:35.949338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6245-6275-46c2-925a-f6db8a0114a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:35.950124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e6245-6275-46c2-925a-f6db8a0114a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:35.950431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e6245-6275-46c2-925a-f6db8a0114a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:35.951900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e6245-6275-46c2-925a-f6db8a0114a1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:35.968974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 497/978] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:35.970578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:36.076743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a90f1c-76a1-4be7-8c06-5fc483e791be None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:36.113698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfce2b4-bb91-4cb1-aacc-9d40d83f4362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:36.114387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce2b4-bb91-4cb1-aacc-9d40d83f4362 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:36.114669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce2b4-bb91-4cb1-aacc-9d40d83f4362 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:36.114943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce2b4-bb91-4cb1-aacc-9d40d83f4362 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:36.115803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfce2b4-bb91-4cb1-aacc-9d40d83f4362 None admin] RBAC: Adding query filter params (role.id=52fd1ff24fe24ad98ca611cd5f52669f, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=70d16599118b4968a18bf3f783a6687f) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:36.116130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfce2b4-bb91-4cb1-aacc-9d40d83f4362 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:36.117595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfce2b4-bb91-4cb1-aacc-9d40d83f4362 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:36.125236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 482/979] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:35 2022] GET /identity/v3/role_assignments?user.id=70d16599118b4968a18bf3f783a6687f&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=52fd1ff24fe24ad98ca611cd5f52669f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:37.046143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f555d93-6593-42fd-ab45-03bebdc9342c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.046685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f555d93-6593-42fd-ab45-03bebdc9342c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.046685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f555d93-6593-42fd-ab45-03bebdc9342c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.048306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 498/980] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:37.053813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e25725-60d3-46e8-8d5b-4d23d54064e2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.054246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e25725-60d3-46e8-8d5b-4d23d54064e2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.054324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e25725-60d3-46e8-8d5b-4d23d54064e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.072018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b7e25725-60d3-46e8-8d5b-4d23d54064e2 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:37.073228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b7e25725-60d3-46e8-8d5b-4d23d54064e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:37.076896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:37.145947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 483/981] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:37.168837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f555d93-6593-42fd-ab45-03bebdc9342c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:37.189501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:37.278345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f555d93-6593-42fd-ab45-03bebdc9342c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:37.312486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7976c-a79c-4f4b-9707-61236169c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:37.313201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7976c-a79c-4f4b-9707-61236169c3ff None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.313477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7976c-a79c-4f4b-9707-61236169c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.313749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7976c-a79c-4f4b-9707-61236169c3ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.314401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7976c-a79c-4f4b-9707-61236169c3ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:37.315865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7976c-a79c-4f4b-9707-61236169c3ff None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a1b7976c-a79c-4f4b-9707-61236169c3ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:37.328074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:37.329048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:37.329915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:37.331675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 499/982] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:37.335354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e25725-60d3-46e8-8d5b-4d23d54064e2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:37.337195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a947c-d19c-48a4-8ee7-232bb61c0944 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.337690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a947c-d19c-48a4-8ee7-232bb61c0944 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.338147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a947c-d19c-48a4-8ee7-232bb61c0944 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.339690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 500/983] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:37.344570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d90007-af55-4b6a-b880-48e6fc03f0ae None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.344854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d90007-af55-4b6a-b880-48e6fc03f0ae None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.345127 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d90007-af55-4b6a-b880-48e6fc03f0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.362204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-80d90007-af55-4b6a-b880-48e6fc03f0ae None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:37.363485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80d90007-af55-4b6a-b880-48e6fc03f0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:37.434614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 501/984] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:37.444461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e25725-60d3-46e8-8d5b-4d23d54064e2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:37.456927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d90007-af55-4b6a-b880-48e6fc03f0ae None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:37.470933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:37.482856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:37.486429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.486429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.486429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.486429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:37.486429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:37.486818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:37.492610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 484/985] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:37.498965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:37.503069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:37.573605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d90007-af55-4b6a-b880-48e6fc03f0ae None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:37.607953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44c334-fec9-4d5c-af64-eb863f8f72e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:37.608355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdbffd-4be3-4e46-9de9-ebe456bdfef6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:37.608653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c334-fec9-4d5c-af64-eb863f8f72e6 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.608928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c334-fec9-4d5c-af64-eb863f8f72e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.609195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c334-fec9-4d5c-af64-eb863f8f72e6 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.609944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44c334-fec9-4d5c-af64-eb863f8f72e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:37.611432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44c334-fec9-4d5c-af64-eb863f8f72e6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:37.635424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 502/986] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:08:37 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:08:37.651851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:37.652660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.652991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.653331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.654128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:37.654519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:37.656043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:37.666974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 485/987] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:37.668370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44c334-fec9-4d5c-af64-eb863f8f72e6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:37.747137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:37.785085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44c334-fec9-4d5c-af64-eb863f8f72e6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:37.821794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cc9de-9542-4aa7-a61a-ccac63accfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:37.822453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc9de-9542-4aa7-a61a-ccac63accfdb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.822744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc9de-9542-4aa7-a61a-ccac63accfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.823023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc9de-9542-4aa7-a61a-ccac63accfdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.823908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cc9de-9542-4aa7-a61a-ccac63accfdb None admin] RBAC: Adding query filter params (interface=admin, service_id=4e64f8d6ec9148669f40818fe78e97e1, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:37.824164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cc9de-9542-4aa7-a61a-ccac63accfdb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:37.825549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cc9de-9542-4aa7-a61a-ccac63accfdb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:37.831166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 503/988] 10.222.0.33 () {56 vars in 1249 bytes} [Thu Jun 23 08:08:37 2022] GET /identity/v3/endpoints?service_id=4e64f8d6ec9148669f40818fe78e97e1&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:37.836600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:37.898773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:37.939382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2038-94a2-4cab-b507-521fa8a25a09 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:37.974210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfee884-f543-431d-9707-940e02958766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:37.974874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee884-f543-431d-9707-940e02958766 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:37.975148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee884-f543-431d-9707-940e02958766 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:37.975456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee884-f543-431d-9707-940e02958766 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:37.976130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee884-f543-431d-9707-940e02958766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:37.977499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee884-f543-431d-9707-940e02958766 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:37.984020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 486/989] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:08:37 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:08:38.173853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fb1cc-8109-4f49-9774-a82efd5193e7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:38.174390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fb1cc-8109-4f49-9774-a82efd5193e7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:38.174390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fb1cc-8109-4f49-9774-a82efd5193e7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:38.175985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 504/990] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:38.181300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938744a-24ac-46e6-9275-7f80e7efb8a8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:38.181772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938744a-24ac-46e6-9275-7f80e7efb8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:38.181852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938744a-24ac-46e6-9275-7f80e7efb8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:38.199953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a938744a-24ac-46e6-9275-7f80e7efb8a8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:38.201139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a938744a-24ac-46e6-9275-7f80e7efb8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:38.270094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 487/991] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:38.292838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fb1cc-8109-4f49-9774-a82efd5193e7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:38.399643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fb1cc-8109-4f49-9774-a82efd5193e7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:38.434367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:38.435079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:38.435395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:38.435672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] PATH_INFO: `/v3/services/image` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:38.436319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:38.438175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:38.439762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:38.451828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:38.452835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:38.453677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:38.455584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 505/992] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 23 08:08:38 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:38.459171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938744a-24ac-46e6-9275-7f80e7efb8a8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:38.541516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:38.586542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938744a-24ac-46e6-9275-7f80e7efb8a8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:38.620747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94019f27-31a0-483f-94e5-6689fa9d5aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:38.621743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019f27-31a0-483f-94e5-6689fa9d5aaa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:38.622023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019f27-31a0-483f-94e5-6689fa9d5aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:38.622295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019f27-31a0-483f-94e5-6689fa9d5aaa None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:38.623054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94019f27-31a0-483f-94e5-6689fa9d5aaa None admin] RBAC: Adding query filter params (name=image) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:38.623387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94019f27-31a0-483f-94e5-6689fa9d5aaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:38.624767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94019f27-31a0-483f-94e5-6689fa9d5aaa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:38.629870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 488/993] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:38.635525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:38.739956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e679b77-b3ca-488c-b88e-49ac1d650b4b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:38.776282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9af08c-1e92-49ac-b3fd-2d625e3b663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:38.776996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9af08c-1e92-49ac-b3fd-2d625e3b663e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:38.777279 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9af08c-1e92-49ac-b3fd-2d625e3b663e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:38.777549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9af08c-1e92-49ac-b3fd-2d625e3b663e None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:38.778253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9af08c-1e92-49ac-b3fd-2d625e3b663e None admin] RBAC: Adding query filter params (type=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:38.778562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9af08c-1e92-49ac-b3fd-2d625e3b663e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:38.779980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9af08c-1e92-49ac-b3fd-2d625e3b663e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:38.784999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 506/994] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:39.672303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4488553-eaaf-477c-8d5e-2d5af6aa7a31 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:39.672850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4488553-eaaf-477c-8d5e-2d5af6aa7a31 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:39.672850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4488553-eaaf-477c-8d5e-2d5af6aa7a31 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:39.674188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 489/995] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:39.679815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca7fa4-240d-48cf-a09f-9e0a586d3227 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:39.680700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca7fa4-240d-48cf-a09f-9e0a586d3227 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:39.680984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca7fa4-240d-48cf-a09f-9e0a586d3227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:39.699329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-78ca7fa4-240d-48cf-a09f-9e0a586d3227 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:39.700548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-78ca7fa4-240d-48cf-a09f-9e0a586d3227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:39.765196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:39.777109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 507/996] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:39.800160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4488553-eaaf-477c-8d5e-2d5af6aa7a31 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:39.913198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4488553-eaaf-477c-8d5e-2d5af6aa7a31 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:39.936714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:39.950228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388af24-a5be-44b3-83e0-797b41d43cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:39.951050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388af24-a5be-44b3-83e0-797b41d43cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:39.951361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388af24-a5be-44b3-83e0-797b41d43cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:39.951639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388af24-a5be-44b3-83e0-797b41d43cc1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:39.952303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388af24-a5be-44b3-83e0-797b41d43cc1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:39.953674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388af24-a5be-44b3-83e0-797b41d43cc1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7388af24-a5be-44b3-83e0-797b41d43cc1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:39.966066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:39.967056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:39.973134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 490/997] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:39.973933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca7fa4-240d-48cf-a09f-9e0a586d3227 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:39.985115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbed47-adae-4135-8731-65663110f203 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:39.985411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbed47-adae-4135-8731-65663110f203 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:39.985687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbed47-adae-4135-8731-65663110f203 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:39.987090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 491/998] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:39.992271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd8d90-b19f-4cc0-923f-476d3930f0f3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:39.992560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd8d90-b19f-4cc0-923f-476d3930f0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:39.992840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd8d90-b19f-4cc0-923f-476d3930f0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.010135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b0cd8d90-b19f-4cc0-923f-476d3930f0f3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:40.011352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b0cd8d90-b19f-4cc0-923f-476d3930f0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:40.075502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca7fa4-240d-48cf-a09f-9e0a586d3227 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:40.087249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 492/999] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:40.108299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd8d90-b19f-4cc0-923f-476d3930f0f3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:40.113032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:40.113706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.113982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.114252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.114970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] RBAC: Adding query filter params (name=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:40.115282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:40.116667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:40.126666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 508/1000] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:40.128496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:40.187781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:40.214418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd8d90-b19f-4cc0-923f-476d3930f0f3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:40.232371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:40.241371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669f6c-3dc8-4cdb-a70e-cd93496ce0ba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:40.251892 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:40.252706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.252983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.253349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.254011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:40.255408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:40.267861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:40.268958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:40.269834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 493/1001] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 08:08:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:40.274891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:40.280057 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:40.280730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.281012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.281287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.281992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:40.282298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:40.283710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:40.293948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 509/1002] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:40.294980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:40.384018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680396fb-bc05-478b-bac3-c46d040be98c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:40.405898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dedc53-5144-4503-9ece-ac5a1bdbab95 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:40.418974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:40.419677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.419961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.420237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.420953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:40.421259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:40.422659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:40.427974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 494/1003] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:40.432926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:40.434129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:40.445319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:40.445994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.446285 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.446565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.447398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] RBAC: Adding query filter params (interface=public, service_id=ca7909602f944b95bbbe148d4f1febe7, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:40.447714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:40.449122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:40.454870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 510/1004] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 08:08:40 2022] GET /identity/v3/endpoints?service_id=ca7909602f944b95bbbe148d4f1febe7&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:40.460432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:40.491147 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:40.552445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7067b-2bf9-4b84-b410-28aee5a4ff54 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:40.563486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca47cb-1779-4304-b6eb-0caadba4c0e1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:40.588857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:40.589636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.589916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.590290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.590984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:40.591316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:40.592798 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:40.597929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 495/1005] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 08:08:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:40.603501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bac89-3a7c-47e9-b7a3-67a94be1875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:40.603662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:40.604189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bac89-3a7c-47e9-b7a3-67a94be1875e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.604474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bac89-3a7c-47e9-b7a3-67a94be1875e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.604747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bac89-3a7c-47e9-b7a3-67a94be1875e None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.605465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bac89-3a7c-47e9-b7a3-67a94be1875e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:40.606865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bac89-3a7c-47e9-b7a3-67a94be1875e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:40.614343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 511/1006] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:08:40 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:08:40.720887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5bed3-733a-4814-9046-cb44a7023593 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:40.731439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:40.762552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b257dc-0d08-4e06-8cf3-bb088093ed19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:40.763254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b257dc-0d08-4e06-8cf3-bb088093ed19 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.763665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b257dc-0d08-4e06-8cf3-bb088093ed19 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.763952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b257dc-0d08-4e06-8cf3-bb088093ed19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.764715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b257dc-0d08-4e06-8cf3-bb088093ed19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:40.766150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b257dc-0d08-4e06-8cf3-bb088093ed19 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:40.782835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 496/1007] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:08:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:08:40.809478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787c031-18ae-41b0-b363-ae6010dc3e9e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.810030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787c031-18ae-41b0-b363-ae6010dc3e9e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.810030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787c031-18ae-41b0-b363-ae6010dc3e9e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.811502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 512/1008] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:40.817948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d0ed-344e-47fb-9710-5188d236b532 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:40.818265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d0ed-344e-47fb-9710-5188d236b532 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:40.818514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d0ed-344e-47fb-9710-5188d236b532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:40.836786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9315d0ed-344e-47fb-9710-5188d236b532 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:40.838015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9315d0ed-344e-47fb-9710-5188d236b532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:40.918223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 497/1009] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:40.941434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787c031-18ae-41b0-b363-ae6010dc3e9e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:40.996985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:41.050099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787c031-18ae-41b0-b363-ae6010dc3e9e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:41.086413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e9fc2-1155-446c-b1f3-0677cdf2f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:41.087098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e9fc2-1155-446c-b1f3-0677cdf2f16d None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:41.087412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e9fc2-1155-446c-b1f3-0677cdf2f16d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:41.087661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e9fc2-1155-446c-b1f3-0677cdf2f16d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:41.088422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e9fc2-1155-446c-b1f3-0677cdf2f16d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:41.089813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e9fc2-1155-446c-b1f3-0677cdf2f16d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:41.104501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 513/1010] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:08:40 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:42.673934 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749f14-34ac-4a5c-af5b-e78da5b67560 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:42.674468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749f14-34ac-4a5c-af5b-e78da5b67560 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:42.674468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749f14-34ac-4a5c-af5b-e78da5b67560 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:42.676293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 498/1011] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:42.681648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffa770-dc38-4e07-855d-a327d85760eb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:42.682073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffa770-dc38-4e07-855d-a327d85760eb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:42.682249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffa770-dc38-4e07-855d-a327d85760eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:42.700001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-afffa770-dc38-4e07-855d-a327d85760eb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:42.701193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-afffa770-dc38-4e07-855d-a327d85760eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:42.778095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 514/1012] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:42 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:42.801052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74749f14-34ac-4a5c-af5b-e78da5b67560 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:42.825461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a2ed5-7f94-4ff8-b02b-9ee5937ecbdd None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:42.825865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a2ed5-7f94-4ff8-b02b-9ee5937ecbdd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:42.826141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a2ed5-7f94-4ff8-b02b-9ee5937ecbdd None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:42.827576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 515/1013] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:42.834240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dc69c-263b-426f-9d35-d12d855ad447 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:42.834504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dc69c-263b-426f-9d35-d12d855ad447 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:42.834816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dc69c-263b-426f-9d35-d12d855ad447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:42.852598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5c3dc69c-263b-426f-9d35-d12d855ad447 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:42.853889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5c3dc69c-263b-426f-9d35-d12d855ad447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:42.884585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:42.903350 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:42.913589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74749f14-34ac-4a5c-af5b-e78da5b67560 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:42.932450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 516/1014] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:42 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:42.952092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:42.952900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:42.953183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:42.953459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:42.954115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:42.955608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:42.955740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dc69c-263b-426f-9d35-d12d855ad447 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:42.967810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:42.968855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 08:08:42.969701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:42.974036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 499/1015] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:42.974699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:43.060689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dc69c-263b-426f-9d35-d12d855ad447 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:43.081397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e4c78-1b6e-494e-b8aa-ed270a8a519e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:43.097792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc086d8-308c-461c-8aaa-bdf4b0b0568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:43.098665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc086d8-308c-461c-8aaa-bdf4b0b0568b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.098952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc086d8-308c-461c-8aaa-bdf4b0b0568b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.099329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc086d8-308c-461c-8aaa-bdf4b0b0568b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.100509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc086d8-308c-461c-8aaa-bdf4b0b0568b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:43.101871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc086d8-308c-461c-8aaa-bdf4b0b0568b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:43.103610 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 517/1016] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:43.109221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc086d8-308c-461c-8aaa-bdf4b0b0568b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:43.118059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:43.118731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.119014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.119318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.120034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:43.120343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:43.122041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:43.131533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 500/1017] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:43.132560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:43.146471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:43.157593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:43.218231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc086d8-308c-461c-8aaa-bdf4b0b0568b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:43.235642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be47b5-c778-492d-b0a4-c12b86d2d990 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:43.251449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eae12a-5232-4b8d-ac40-068fde6aa0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:43.252125 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eae12a-5232-4b8d-ac40-068fde6aa0c5 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.252378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eae12a-5232-4b8d-ac40-068fde6aa0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.252650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eae12a-5232-4b8d-ac40-068fde6aa0c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.253390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eae12a-5232-4b8d-ac40-068fde6aa0c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:43.254834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eae12a-5232-4b8d-ac40-068fde6aa0c5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:43.268705 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0305ad8-2d25-4637-b97e-472e8bec4df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:43.269394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0305ad8-2d25-4637-b97e-472e8bec4df2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.269694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0305ad8-2d25-4637-b97e-472e8bec4df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.269975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0305ad8-2d25-4637-b97e-472e8bec4df2 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.270693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0305ad8-2d25-4637-b97e-472e8bec4df2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:43.271121 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0305ad8-2d25-4637-b97e-472e8bec4df2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:43.272439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0305ad8-2d25-4637-b97e-472e8bec4df2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:43.277902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 501/1018] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 08:08:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:43.279135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674bd2c-02bd-4c37-ac1c-5230a3da448a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.279527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674bd2c-02bd-4c37-ac1c-5230a3da448a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.279765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674bd2c-02bd-4c37-ac1c-5230a3da448a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.281002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 502/1019] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:43.284727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 518/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:08:43 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:08:43.285772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674bd2c-02bd-4c37-ac1c-5230a3da448a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:43.286378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5ea5b-a946-4334-aca6-cbdfdbd75a1b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.286668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5ea5b-a946-4334-aca6-cbdfdbd75a1b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.286947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5ea5b-a946-4334-aca6-cbdfdbd75a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.313154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e6d5ea5b-a946-4334-aca6-cbdfdbd75a1b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:43.314839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e6d5ea5b-a946-4334-aca6-cbdfdbd75a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:43.392174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:43.403256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 519/1021] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:43 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:43.421999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674bd2c-02bd-4c37-ac1c-5230a3da448a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:43.423302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5ea5b-a946-4334-aca6-cbdfdbd75a1b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:43.442874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:43.463899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bb102-025c-4a1b-b9cf-c3d782aa9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:43.464821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb102-025c-4a1b-b9cf-c3d782aa9647 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.464921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb102-025c-4a1b-b9cf-c3d782aa9647 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.465169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb102-025c-4a1b-b9cf-c3d782aa9647 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.465961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bb102-025c-4a1b-b9cf-c3d782aa9647 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:43.467392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bb102-025c-4a1b-b9cf-c3d782aa9647 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:43.493823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 503/1022] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:08:43 2022] POST /identity/v3/endpoints => generated 334 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:43.549424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5ea5b-a946-4334-aca6-cbdfdbd75a1b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:43.597530 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:43.598301 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.598586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.598853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] PATH_INFO: `/v3/services/image` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.599559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:43.601060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:43.614270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:43.615398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:43.616730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 520/1023] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 23 08:08:43 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:43.621978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bb102-025c-4a1b-b9cf-c3d782aa9647 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:43.744374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bb102-025c-4a1b-b9cf-c3d782aa9647 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:43.781796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:43.782460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.782743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.783019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.783891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:43.784219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:43.785613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:43.791701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 504/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:43.797252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:43.886350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:43.916042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4128022-051a-4b20-9ef6-d339da54f734 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:43.956367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:43.957109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:43.957413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:43.957688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:43.958389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] RBAC: Adding query filter params (type=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:43.958692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:43.960116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:43.965711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 521/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:43.971562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:43.984217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:44.089236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c36629-27e1-4374-b931-65060c4b1dd7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:44.129142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88271ffb-df39-460a-9a15-ff97e33f0b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:44.129816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88271ffb-df39-460a-9a15-ff97e33f0b6c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:44.130095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88271ffb-df39-460a-9a15-ff97e33f0b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:44.130365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88271ffb-df39-460a-9a15-ff97e33f0b6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:44.131157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88271ffb-df39-460a-9a15-ff97e33f0b6c None admin] RBAC: Adding query filter params (interface=public, service_id=c36f2fa3157a4178a2f4c3bdf613a40a, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:44.131504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88271ffb-df39-460a-9a15-ff97e33f0b6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:44.132888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88271ffb-df39-460a-9a15-ff97e33f0b6c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:44.138803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 505/1026] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 08:08:43 2022] GET /identity/v3/endpoints?service_id=c36f2fa3157a4178a2f4c3bdf613a40a&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:44.144426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:44.256291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f770f8b-1ef2-40de-952a-2faf2006a087 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:44.294773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6d205-7de7-4f62-bc7e-da3dd7c3d6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:44.295470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6d205-7de7-4f62-bc7e-da3dd7c3d6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:44.295748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6d205-7de7-4f62-bc7e-da3dd7c3d6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:44.295999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6d205-7de7-4f62-bc7e-da3dd7c3d6f1 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:44.296818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6d205-7de7-4f62-bc7e-da3dd7c3d6f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:44.298284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6d205-7de7-4f62-bc7e-da3dd7c3d6f1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:44.307537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 522/1027] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:08:44 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:08:45.381372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655275-de48-4df6-914f-891f6ce443a8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:45.381927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655275-de48-4df6-914f-891f6ce443a8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:45.381927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655275-de48-4df6-914f-891f6ce443a8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:45.383351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 506/1028] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:45.388732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4ba48-69d5-4f6c-b365-1c4c0d09c791 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:45.389177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4ba48-69d5-4f6c-b365-1c4c0d09c791 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:45.389346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4ba48-69d5-4f6c-b365-1c4c0d09c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:45.394732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:45.407106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-36f4ba48-69d5-4f6c-b365-1c4c0d09c791 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:45.408485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-36f4ba48-69d5-4f6c-b365-1c4c0d09c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:45.481108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 523/1029] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:45.511481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655275-de48-4df6-914f-891f6ce443a8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:45.589266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:45.622648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655275-de48-4df6-914f-891f6ce443a8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:45.659733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dda07-21cc-4131-aa1a-1dec2cff812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:45.660457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dda07-21cc-4131-aa1a-1dec2cff812e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:45.660739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dda07-21cc-4131-aa1a-1dec2cff812e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:45.661011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dda07-21cc-4131-aa1a-1dec2cff812e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:45.662176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dda07-21cc-4131-aa1a-1dec2cff812e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:45.663661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dda07-21cc-4131-aa1a-1dec2cff812e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:45.669798 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 507/1030] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:45.669937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba48f6d-bc1d-4458-8857-e230e99bcee0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:45.669937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba48f6d-bc1d-4458-8857-e230e99bcee0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:45.669937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba48f6d-bc1d-4458-8857-e230e99bcee0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:45.669937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 524/1031] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:45.671800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dda07-21cc-4131-aa1a-1dec2cff812e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:45.674306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d1fc-de41-4c9c-91cc-510915d72ea4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:45.674579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d1fc-de41-4c9c-91cc-510915d72ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:45.674913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d1fc-de41-4c9c-91cc-510915d72ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:45.691087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-63e3d1fc-de41-4c9c-91cc-510915d72ea4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:45.692288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-63e3d1fc-de41-4c9c-91cc-510915d72ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:45.766959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 525/1032] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:45.778134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dda07-21cc-4131-aa1a-1dec2cff812e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:45.790552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3d1fc-de41-4c9c-91cc-510915d72ea4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:45.816727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8434c1-2a90-49b0-9018-bac546222ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:45.817720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8434c1-2a90-49b0-9018-bac546222ded None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:45.818194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8434c1-2a90-49b0-9018-bac546222ded None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:45.818757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8434c1-2a90-49b0-9018-bac546222ded None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:45.819786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8434c1-2a90-49b0-9018-bac546222ded None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:45.821530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8434c1-2a90-49b0-9018-bac546222ded None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:45.831241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:45.847725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 508/1033] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:08:45 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:45.856687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:45.901506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3d1fc-de41-4c9c-91cc-510915d72ea4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:45.940268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddd78d-fc13-4f62-8aae-331f98855e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:45.941087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddd78d-fc13-4f62-8aae-331f98855e5a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:45.941368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddd78d-fc13-4f62-8aae-331f98855e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:45.941640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddd78d-fc13-4f62-8aae-331f98855e5a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:45.953524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddd78d-fc13-4f62-8aae-331f98855e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e9ddd78d-fc13-4f62-8aae-331f98855e5a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:45.962217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:45.963296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:45.969661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 526/1034] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 08:08:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:45.970406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8434c1-2a90-49b0-9018-bac546222ded None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.077566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8434c1-2a90-49b0-9018-bac546222ded None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.112098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.113283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.113764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.114236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.115271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:46.115789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:46.117408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:46.126660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 509/1035] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 08:08:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:46.132500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddd78d-fc13-4f62-8aae-331f98855e5a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.238683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddd78d-fc13-4f62-8aae-331f98855e5a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.262924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:46.277135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4beec-6062-454c-8f41-d2792fe6dd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.277954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4beec-6062-454c-8f41-d2792fe6dd94 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.278238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4beec-6062-454c-8f41-d2792fe6dd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.278513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4beec-6062-454c-8f41-d2792fe6dd94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.290813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4beec-6062-454c-8f41-d2792fe6dd94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:46.292336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4beec-6062-454c-8f41-d2792fe6dd94 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4cf4beec-6062-454c-8f41-d2792fe6dd94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:46.310298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:46.311541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:46.312475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:46.312475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:46.312475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:46.312475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:46.312475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 527/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:46.318734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.343349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32738a9-1242-439c-9f54-6acbb5172fc2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.343655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32738a9-1242-439c-9f54-6acbb5172fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.344218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32738a9-1242-439c-9f54-6acbb5172fc2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.345409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 528/1037] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:46.350065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036a72-f3ce-4a93-8954-f6760fdc887c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.350350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036a72-f3ce-4a93-8954-f6760fdc887c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.350625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036a72-f3ce-4a93-8954-f6760fdc887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.370190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e5036a72-f3ce-4a93-8954-f6760fdc887c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:46.371715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e5036a72-f3ce-4a93-8954-f6760fdc887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:46.386802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:46.434903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72383df7-ac15-486a-9a60-3d015710d5a3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.442380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 529/1038] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:46.465512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5036a72-f3ce-4a93-8954-f6760fdc887c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.471662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.472337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.472598 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.472866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.473612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:46.473916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:46.475387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:46.490400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 510/1039] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:46.495562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.576224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5036a72-f3ce-4a93-8954-f6760fdc887c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.578522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:46.602727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e4de4-1860-45a7-887d-633a6a58a9a2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.611340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.612218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.612519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.612794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] PATH_INFO: `/v3/services/image` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.613442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:46.614826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:46.619794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:46.627398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:46.628506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:46.629525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 530/1040] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 23 08:08:46 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:46.634371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.642839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.643843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.644120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.644385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.651903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:46.653346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:46.668537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:46.669726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:46.671308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 511/1041] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:08:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:46.675335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.739735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c0886-986f-4293-aaf9-9b654e09742a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.774517 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.775130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.775408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.775684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.776507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:46.776789 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:46.778197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:46.778875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcde1b3-464f-4ed8-835e-5c4f0b8cc21d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.783302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 531/1042] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:46.789077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.818900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.819619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.820087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.820196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.820915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:46.821025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:46.821297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:46.822723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:46.841457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 512/1043] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:08:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:46.846911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.881385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:46.898042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20bb60-e7c0-4e63-ab13-684c30daa5c1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.934179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.935116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.935568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.936002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:46.936718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] RBAC: Adding query filter params (type=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:46.937020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:46.938416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:46.947981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 532/1044] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:46.948907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:46.955732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d000b-95b7-4394-8bd9-e3bb9007ace3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:46.995084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa0264-5233-4533-bf49-31317baf0392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:46.995780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa0264-5233-4533-bf49-31317baf0392 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:46.996372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa0264-5233-4533-bf49-31317baf0392 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:46.996646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa0264-5233-4533-bf49-31317baf0392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:47.064537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:47.072247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e34d8-e84d-4239-b6f7-1398da7a4f14 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:47.078105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa0264-5233-4533-bf49-31317baf0392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:47.080265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa0264-5233-4533-bf49-31317baf0392 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:47.115249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91e4f-a913-4228-beb4-cf8444fe9f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:47.116080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91e4f-a913-4228-beb4-cf8444fe9f3e None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:47.116460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91e4f-a913-4228-beb4-cf8444fe9f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:47.116742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91e4f-a913-4228-beb4-cf8444fe9f3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:47.117488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb91e4f-a913-4228-beb4-cf8444fe9f3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:47.119607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb91e4f-a913-4228-beb4-cf8444fe9f3e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:47.126311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:47.155290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 533/1045] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:08:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:47.225193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 513/1046] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:46 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 08:08:47.230621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91e4f-a913-4228-beb4-cf8444fe9f3e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:47.368941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91e4f-a913-4228-beb4-cf8444fe9f3e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:47.413136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:47.414350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:47.414849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:47.415394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:47.426965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:47.428758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:47.438656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:47.452678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:47.454044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:47.454941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:47.454941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:47.454941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:47.454941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:47.454941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 534/1047] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:47.461544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa0264-5233-4533-bf49-31317baf0392 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:47.586464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa0264-5233-4533-bf49-31317baf0392 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:47.621982 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:47.622556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:47.622832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:47.623102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:47.623856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:47.624160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:47.625530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:47.627145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:47.639018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 514/1048] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:47.644812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:47.751846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a318-47dd-49b5-86d6-b7851b6423eb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:47.788111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a062edf-7fab-4dd0-817d-5fd4d0743196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:47.789734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062edf-7fab-4dd0-817d-5fd4d0743196 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:47.790400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062edf-7fab-4dd0-817d-5fd4d0743196 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:47.791074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062edf-7fab-4dd0-817d-5fd4d0743196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:47.801300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a062edf-7fab-4dd0-817d-5fd4d0743196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:47.804008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a062edf-7fab-4dd0-817d-5fd4d0743196 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3a062edf-7fab-4dd0-817d-5fd4d0743196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:47.824549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:47.825627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:47.826522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:47.826522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:47.826522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:47.826522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:47.826522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:47.826522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:47.826522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:47.826522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:47.827928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 535/1049] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:47.833823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:47.950053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ba3e2-6acb-423b-84e4-3052a3521732 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:47.954470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af82b4-b2c5-4b29-b20c-edb443458541 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:47.954731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af82b4-b2c5-4b29-b20c-edb443458541 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:47.955032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af82b4-b2c5-4b29-b20c-edb443458541 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:47.956958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 536/1050] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:47.962050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abeebd-8c4f-44d4-9080-6a139279de73 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:47.962304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abeebd-8c4f-44d4-9080-6a139279de73 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:47.962610 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abeebd-8c4f-44d4-9080-6a139279de73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:47.982172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c1abeebd-8c4f-44d4-9080-6a139279de73 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:47.983519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c1abeebd-8c4f-44d4-9080-6a139279de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:47.985680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:47.987321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:47.987520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:47.987748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:47.988546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:47.988852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:47.990280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:48.004027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 515/1051] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:48.009834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:48.042005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:48.059378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 537/1052] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:48.069523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:48.083523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abeebd-8c4f-44d4-9080-6a139279de73 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:48.116221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2adf37-a950-4ee0-9fa2-d35347db9bdc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:48.152983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:48.153746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:48.154096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:48.154438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:48.155386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] RBAC: Adding query filter params (role.id=52fd1ff24fe24ad98ca611cd5f52669f, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=96456bfed82d469b888bd495ec46e796) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:48.155769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:48.157249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:48.164363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 516/1053] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:48 2022] GET /identity/v3/role_assignments?user.id=96456bfed82d469b888bd495ec46e796&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=52fd1ff24fe24ad98ca611cd5f52669f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:48.199541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abeebd-8c4f-44d4-9080-6a139279de73 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:48.240406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90656157-fa3b-42fb-bde7-36be4f39c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:48.241142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90656157-fa3b-42fb-bde7-36be4f39c700 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:48.241427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90656157-fa3b-42fb-bde7-36be4f39c700 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:48.241704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90656157-fa3b-42fb-bde7-36be4f39c700 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:48.254640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90656157-fa3b-42fb-bde7-36be4f39c700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-90656157-fa3b-42fb-bde7-36be4f39c700 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:48.262713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:48.263778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:48.271079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 538/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:48.271866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:48.389312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df2c95-82bf-4e78-a0f8-b467e6adec7a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:48.408146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:48.428027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:48.428705 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:48.428991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:48.429269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:48.430016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:48.430327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:48.431745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:48.441258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 517/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:48.446896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90656157-fa3b-42fb-bde7-36be4f39c700 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:48.507160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:48.563981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90656157-fa3b-42fb-bde7-36be4f39c700 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:48.598752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f5389-747f-4d0b-8d2a-27ab4b67d70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:48.599665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f5389-747f-4d0b-8d2a-27ab4b67d70f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:48.599950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f5389-747f-4d0b-8d2a-27ab4b67d70f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:48.600226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f5389-747f-4d0b-8d2a-27ab4b67d70f None admin] PATH_INFO: `/v3/users/3b2e25f4b7e04d688beb57e7b92ada26` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:48.616294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f5389-747f-4d0b-8d2a-27ab4b67d70f None admin] RBAC: Authorizing `identity:get_user(user_id=3b2e25f4b7e04d688beb57e7b92ada26)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:48.617758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f5389-747f-4d0b-8d2a-27ab4b67d70f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:48.619697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 539/1056] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:08:48 2022] GET /identity/v3/users/3b2e25f4b7e04d688beb57e7b92ada26 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:48.625799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:48.737499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd1ba1-3f58-408d-b73d-74b73f348545 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:48.775161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78b05a-e2d1-46fb-86ce-d7b474f931ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:48.775759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78b05a-e2d1-46fb-86ce-d7b474f931ef None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:48.776042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78b05a-e2d1-46fb-86ce-d7b474f931ef None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:48.776317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78b05a-e2d1-46fb-86ce-d7b474f931ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:48.826682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:48.849050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78b05a-e2d1-46fb-86ce-d7b474f931ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:48.850508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78b05a-e2d1-46fb-86ce-d7b474f931ef None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:48.952317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 518/1057] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:08:48.958648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f5389-747f-4d0b-8d2a-27ab4b67d70f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:49.062631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f5389-747f-4d0b-8d2a-27ab4b67d70f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:49.072061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:49.100396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0260e-1dd2-4944-b9a8-ef660796a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:49.101263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0260e-1dd2-4944-b9a8-ef660796a04e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:49.101538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0260e-1dd2-4944-b9a8-ef660796a04e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:49.101806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0260e-1dd2-4944-b9a8-ef660796a04e None admin] PATH_INFO: `/v3/projects/f7ff831214604ae4954c77bec590ddc6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:49.102966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0260e-1dd2-4944-b9a8-ef660796a04e None admin] RBAC: Authorizing `identity:get_project(project_id=f7ff831214604ae4954c77bec590ddc6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:49.104457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0260e-1dd2-4944-b9a8-ef660796a04e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:49.111160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 540/1058] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:08:48 2022] GET /identity/v3/projects/f7ff831214604ae4954c77bec590ddc6 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:49.112634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78b05a-e2d1-46fb-86ce-d7b474f931ef None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:49.197956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:49.218828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78b05a-e2d1-46fb-86ce-d7b474f931ef None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:49.239755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623760f2-c9bb-476f-ba71-a77df1162998 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:49.240074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623760f2-c9bb-476f-ba71-a77df1162998 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:49.240315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623760f2-c9bb-476f-ba71-a77df1162998 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:49.241759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 541/1059] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:49.248273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76f11-4fda-4753-a557-dec083ded77c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:49.248583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76f11-4fda-4753-a557-dec083ded77c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:49.248829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76f11-4fda-4753-a557-dec083ded77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:49.258708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:49.259766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:49.260261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:49.260564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:49.261390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=f7ff831214604ae4954c77bec590ddc6, user.id=3b2e25f4b7e04d688beb57e7b92ada26) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:49.261687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:49.263110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:49.266041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-18c76f11-4fda-4753-a557-dec083ded77c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:49.267224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-18c76f11-4fda-4753-a557-dec083ded77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:49.269769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 519/1060] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:49 2022] GET /identity/v3/role_assignments?user.id=3b2e25f4b7e04d688beb57e7b92ada26&scope.project.id=f7ff831214604ae4954c77bec590ddc6&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:49.359631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 542/1061] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:49.376453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:49.485438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67cac0-f900-4334-a9bf-205276fdf75e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:49.526084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba1057-e6f3-46e4-9cb0-cdb69232a822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:49.526743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba1057-e6f3-46e4-9cb0-cdb69232a822 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:49.527023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba1057-e6f3-46e4-9cb0-cdb69232a822 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:49.527319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba1057-e6f3-46e4-9cb0-cdb69232a822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:49.541870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:49.595078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba1057-e6f3-46e4-9cb0-cdb69232a822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:49.596624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba1057-e6f3-46e4-9cb0-cdb69232a822 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:49.697333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 520/1062] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:08:49.703342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c76f11-4fda-4753-a557-dec083ded77c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:49.804001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c76f11-4fda-4753-a557-dec083ded77c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:49.811542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:49.839587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b01c8-0faa-489a-ab28-dc4a861e795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:49.840700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b01c8-0faa-489a-ab28-dc4a861e795d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:49.841153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b01c8-0faa-489a-ab28-dc4a861e795d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:49.841594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b01c8-0faa-489a-ab28-dc4a861e795d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:49.852104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b01c8-0faa-489a-ab28-dc4a861e795d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:49.853755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b01c8-0faa-489a-ab28-dc4a861e795d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-834b01c8-0faa-489a-ab28-dc4a861e795d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:49.872077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:49.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:49.874005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:49.874005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:49.874005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:49.874005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:49.879786 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 543/1063] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:49.881408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba1057-e6f3-46e4-9cb0-cdb69232a822 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:49.929355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:49.984863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba1057-e6f3-46e4-9cb0-cdb69232a822 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:50.018972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be787769-2aa4-4426-8870-f562539b00d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:50.019663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be787769-2aa4-4426-8870-f562539b00d3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:50.019941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be787769-2aa4-4426-8870-f562539b00d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:50.020208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be787769-2aa4-4426-8870-f562539b00d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:50.021014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be787769-2aa4-4426-8870-f562539b00d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:50.021318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be787769-2aa4-4426-8870-f562539b00d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:50.022673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be787769-2aa4-4426-8870-f562539b00d3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:50.039322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 521/1064] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:50.167262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec4700-19bc-4472-89c9-4a41fc66ceca None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:50.167661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec4700-19bc-4472-89c9-4a41fc66ceca None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:50.167827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec4700-19bc-4472-89c9-4a41fc66ceca None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:50.169418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 544/1065] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:50.175198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848b380-544c-430f-8f34-3f5598b30aa5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:50.175479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848b380-544c-430f-8f34-3f5598b30aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:50.175786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848b380-544c-430f-8f34-3f5598b30aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:50.193616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4848b380-544c-430f-8f34-3f5598b30aa5 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:50.194825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4848b380-544c-430f-8f34-3f5598b30aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:50.266568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 522/1066] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:50.289209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec4700-19bc-4472-89c9-4a41fc66ceca None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:50.396693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec4700-19bc-4472-89c9-4a41fc66ceca None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:50.429889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773cf27-247b-4d57-a5c2-396948555d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:50.430696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773cf27-247b-4d57-a5c2-396948555d83 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:50.430973 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773cf27-247b-4d57-a5c2-396948555d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:50.431252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773cf27-247b-4d57-a5c2-396948555d83 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:50.441379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773cf27-247b-4d57-a5c2-396948555d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6773cf27-247b-4d57-a5c2-396948555d83 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:50.449516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:50.450453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:50.451343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 545/1067] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 08:08:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:50.457365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848b380-544c-430f-8f34-3f5598b30aa5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:50.491385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:50.561318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848b380-544c-430f-8f34-3f5598b30aa5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:50.595789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:50.596507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:50.596744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:50.597052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:50.597764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:50.598072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:50.599481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:50.607662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 523/1068] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 08:08:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:50.613559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773cf27-247b-4d57-a5c2-396948555d83 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:50.633818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:50.714838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773cf27-247b-4d57-a5c2-396948555d83 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:50.749855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:50.750660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:50.750943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:50.751245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:50.762209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:50.763710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:50.780857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:50.781787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:50.783262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:50.783262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:50.783262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:50.783262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:50.783262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 546/1069] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:50.788113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:50.882754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:50.889200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a34d7-90da-434f-9f9c-1bdeaf450568 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:50.923809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:50.924472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:50.924748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:50.925018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:50.925743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:50.926044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:50.927456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:50.939900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 524/1070] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:50.945758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.019840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:51.052471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a860422-3f9f-4b97-9d6c-72a581ce9fcb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.087098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4593e3-58d2-4edf-b18e-73cbd21b7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.088037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4593e3-58d2-4edf-b18e-73cbd21b7136 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.088318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4593e3-58d2-4edf-b18e-73cbd21b7136 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.088595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4593e3-58d2-4edf-b18e-73cbd21b7136 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.096342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4593e3-58d2-4edf-b18e-73cbd21b7136 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:51.097995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4593e3-58d2-4edf-b18e-73cbd21b7136 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7e4593e3-58d2-4edf-b18e-73cbd21b7136 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:51.112296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:51.113481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:51.114609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:51.118204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 547/1071] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:08:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:51.119596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.229205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdeeea-6667-4cb7-a2b7-52c45756712d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.265028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.265691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.265965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.266235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.267053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:51.267390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:51.268937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:51.271581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9ccdc-b93f-4cb1-bf80-03e9d36f5d91 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.271874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9ccdc-b93f-4cb1-bf80-03e9d36f5d91 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.272152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9ccdc-b93f-4cb1-bf80-03e9d36f5d91 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.273534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 548/1072] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:51.278583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94e1c-bf66-4820-aba9-4c6c99d15e0c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.278869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94e1c-bf66-4820-aba9-4c6c99d15e0c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.279145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94e1c-bf66-4820-aba9-4c6c99d15e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.290984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 525/1073] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:51.292939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.296682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-03d94e1c-bf66-4820-aba9-4c6c99d15e0c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:51.297881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-03d94e1c-bf66-4820-aba9-4c6c99d15e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:51.308088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:51.366988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 549/1074] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:51.390221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94e1c-bf66-4820-aba9-4c6c99d15e0c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.392929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22344a-62c4-4926-86d3-233db0e485f6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.427439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.428235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.428517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.428786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.438851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:51.440363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:51.448589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:51.458826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:51.459907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:51.465747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:51.465747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:51.465747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:51.465747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:51.465747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 526/1075] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:51.465747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.489950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94e1c-bf66-4820-aba9-4c6c99d15e0c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.527953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115625fa-e49c-4470-a782-e1c37b47203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.528753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115625fa-e49c-4470-a782-e1c37b47203e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.529030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115625fa-e49c-4470-a782-e1c37b47203e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.529282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115625fa-e49c-4470-a782-e1c37b47203e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.538657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:51.539281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115625fa-e49c-4470-a782-e1c37b47203e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-115625fa-e49c-4470-a782-e1c37b47203e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:51.546826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:51.548015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:51.548871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 550/1076] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:51.554553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115625fa-e49c-4470-a782-e1c37b47203e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.575711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e0e85-0c79-4144-a43e-a8567e8d7eb3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.609309 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.609968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.610248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.610657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.611407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:51.611715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:51.613345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:51.626022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 527/1077] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:51.631888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.661242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115625fa-e49c-4470-a782-e1c37b47203e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.680672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:51.694756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.695465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.695746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.696021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.696768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:51.697078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:51.698473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:51.706644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 551/1078] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:51.712452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.736582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5026cc-9fd6-4b78-97cc-977ceb1102f5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.763771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:51.773428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.774275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.774551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.774826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.781681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:51.783133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:51.797256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:51.798801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:51.799918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 528/1079] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:51.804443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.816284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48a285-47b7-4f27-924c-083169eef807 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.852781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761736dd-f6ab-4a79-9623-8b6e9edef4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.853673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761736dd-f6ab-4a79-9623-8b6e9edef4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.853950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761736dd-f6ab-4a79-9623-8b6e9edef4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.854223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761736dd-f6ab-4a79-9623-8b6e9edef4f5 None admin] PATH_INFO: `/v3/users/3b2e25f4b7e04d688beb57e7b92ada26` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.855471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761736dd-f6ab-4a79-9623-8b6e9edef4f5 None admin] RBAC: Authorizing `identity:get_user(user_id=3b2e25f4b7e04d688beb57e7b92ada26)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:51.856929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761736dd-f6ab-4a79-9623-8b6e9edef4f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:51.858611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 552/1080] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/users/3b2e25f4b7e04d688beb57e7b92ada26 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:51.864348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761736dd-f6ab-4a79-9623-8b6e9edef4f5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:51.910724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ed8fa-f784-41dc-9120-210ea36e0317 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.914926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:51.947425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:51.948080 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:51.948355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:51.948599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:51.949408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:51.949711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:51.951141 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:51.967866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 529/1081] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:51.968051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761736dd-f6ab-4a79-9623-8b6e9edef4f5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:51.969215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:52.000420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e00c02-431c-4757-9397-0fd838e78c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:52.001344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00c02-431c-4757-9397-0fd838e78c88 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:52.001611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00c02-431c-4757-9397-0fd838e78c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:52.001884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00c02-431c-4757-9397-0fd838e78c88 None admin] PATH_INFO: `/v3/projects/f7ff831214604ae4954c77bec590ddc6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:52.003200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e00c02-431c-4757-9397-0fd838e78c88 None admin] RBAC: Authorizing `identity:get_project(project_id=f7ff831214604ae4954c77bec590ddc6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:52.004608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e00c02-431c-4757-9397-0fd838e78c88 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:52.006371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 553/1082] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:08:51 2022] GET /identity/v3/projects/f7ff831214604ae4954c77bec590ddc6 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:52.012255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e00c02-431c-4757-9397-0fd838e78c88 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:52.018398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:52.072424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b33e2-4293-4265-8d8b-5a5f5aa54b26 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:52.104711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263b1ba-0e6d-42b4-911a-a1007be6447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:52.105607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b1ba-0e6d-42b4-911a-a1007be6447f None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:52.105896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b1ba-0e6d-42b4-911a-a1007be6447f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:52.106155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b1ba-0e6d-42b4-911a-a1007be6447f None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/96456bfed82d469b888bd495ec46e796/roles/52fd1ff24fe24ad98ca611cd5f52669f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:52.108365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263b1ba-0e6d-42b4-911a-a1007be6447f None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=96456bfed82d469b888bd495ec46e796, role_id=52fd1ff24fe24ad98ca611cd5f52669f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:52.109804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263b1ba-0e6d-42b4-911a-a1007be6447f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:52.111827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e00c02-431c-4757-9397-0fd838e78c88 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:52.123953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 530/1083] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:08:51 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/96456bfed82d469b888bd495ec46e796/roles/52fd1ff24fe24ad98ca611cd5f52669f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:08:52.130563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:52.149491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66929ebe-59bc-4826-b8b8-d695a5d3e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:52.150425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929ebe-59bc-4826-b8b8-d695a5d3e773 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:52.150704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929ebe-59bc-4826-b8b8-d695a5d3e773 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:52.150981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929ebe-59bc-4826-b8b8-d695a5d3e773 None admin] PATH_INFO: `/v3/projects/f7ff831214604ae4954c77bec590ddc6/users/3b2e25f4b7e04d688beb57e7b92ada26/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:52.153229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66929ebe-59bc-4826-b8b8-d695a5d3e773 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7ff831214604ae4954c77bec590ddc6, user_id=3b2e25f4b7e04d688beb57e7b92ada26, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:52.154932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66929ebe-59bc-4826-b8b8-d695a5d3e773 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:52.170889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 554/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:08:52 2022] PUT /identity/v3/projects/f7ff831214604ae4954c77bec590ddc6/users/3b2e25f4b7e04d688beb57e7b92ada26/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:08:54.128349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b450-639e-437e-a646-4e0e1693f5a8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.128918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b450-639e-437e-a646-4e0e1693f5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.128918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b450-639e-437e-a646-4e0e1693f5a8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.130259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 531/1085] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:54.135527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81562f1-e000-423e-85ea-4651b5999e63 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.135975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81562f1-e000-423e-85ea-4651b5999e63 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.136085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81562f1-e000-423e-85ea-4651b5999e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.153361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b81562f1-e000-423e-85ea-4651b5999e63 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:54.154550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b81562f1-e000-423e-85ea-4651b5999e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:54.167803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f36be9-32c0-435e-9dab-4b744e69a330 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.168088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f36be9-32c0-435e-9dab-4b744e69a330 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.168371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f36be9-32c0-435e-9dab-4b744e69a330 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.169691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 532/1086] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:08:54.174683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45636e-1e49-47f5-89b2-c79087d82f19 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.175000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45636e-1e49-47f5-89b2-c79087d82f19 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.175237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45636e-1e49-47f5-89b2-c79087d82f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.193310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5d45636e-1e49-47f5-89b2-c79087d82f19 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:54.194516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5d45636e-1e49-47f5-89b2-c79087d82f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:54.226901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 555/1087] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:54.251336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81562f1-e000-423e-85ea-4651b5999e63 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.267683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 533/1088] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:08:54.291354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45636e-1e49-47f5-89b2-c79087d82f19 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.325542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:54.348770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81562f1-e000-423e-85ea-4651b5999e63 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:54.358219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:54.388580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f00226-b330-487b-ac43-d53e28053f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:54.389334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00226-b330-487b-ac43-d53e28053f36 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.389616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00226-b330-487b-ac43-d53e28053f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.389888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00226-b330-487b-ac43-d53e28053f36 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.394734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45636e-1e49-47f5-89b2-c79087d82f19 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:54.400481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f00226-b330-487b-ac43-d53e28053f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f6f00226-b330-487b-ac43-d53e28053f36 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:54.408800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:54.409751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:54.410800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 556/1089] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:54.417368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f00226-b330-487b-ac43-d53e28053f36 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.433004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628385a-a98b-497d-9354-a2b381ac7f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:54.433807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628385a-a98b-497d-9354-a2b381ac7f78 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.434093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628385a-a98b-497d-9354-a2b381ac7f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.434367 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628385a-a98b-497d-9354-a2b381ac7f78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.444714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628385a-a98b-497d-9354-a2b381ac7f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-1628385a-a98b-497d-9354-a2b381ac7f78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:08:54.452733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:54.453905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:54.454880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 534/1090] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:54.460055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628385a-a98b-497d-9354-a2b381ac7f78 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.522497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f00226-b330-487b-ac43-d53e28053f36 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:54.556061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:54.556782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:54.557571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.557984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.558309 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.559113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:54.559454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:54.560860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:54.562354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628385a-a98b-497d-9354-a2b381ac7f78 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:54.569530 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 557/1091] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:54.575085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.601067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:54.601734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.602012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.602282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.603028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:54.603363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:54.604737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:54.606038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:54.613184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 535/1092] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:54.618970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.679309 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb4859-c6e1-42f7-818b-bcd4bba1e147 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:54.716290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:54.716832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.717110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.717383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.721997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ddcf8-da3b-47cb-92f3-a5596c30fa65 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:54.728769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:54.730256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:54.748100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:54.749031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:54.750179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:54.750179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:54.750179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:54.750179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:54.750179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 558/1093] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:54.755061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.757297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb15d90-8bbe-4e77-9534-0e1ccf9ce7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:54.758285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb15d90-8bbe-4e77-9534-0e1ccf9ce7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.758568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb15d90-8bbe-4e77-9534-0e1ccf9ce7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.758845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb15d90-8bbe-4e77-9534-0e1ccf9ce7c8 None admin] PATH_INFO: `/v3/users/3b2e25f4b7e04d688beb57e7b92ada26` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.760187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb15d90-8bbe-4e77-9534-0e1ccf9ce7c8 None admin] RBAC: Authorizing `identity:get_user(user_id=3b2e25f4b7e04d688beb57e7b92ada26)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:54.761621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb15d90-8bbe-4e77-9534-0e1ccf9ce7c8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:54.763279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 536/1094] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/users/3b2e25f4b7e04d688beb57e7b92ada26 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:54.769508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb15d90-8bbe-4e77-9534-0e1ccf9ce7c8 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.797741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:54.838126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:54.856921 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5c51-caec-4480-a8a5-801f2d4f793d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:54.869564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb15d90-8bbe-4e77-9534-0e1ccf9ce7c8 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:54.892112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:54.892773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.893037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.893425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.894161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:54.894462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:54.895891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:54.903868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078a439-586e-4acb-9535-33b17f308fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:54.904495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a439-586e-4acb-9535-33b17f308fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:54.904774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a439-586e-4acb-9535-33b17f308fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:54.905046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a439-586e-4acb-9535-33b17f308fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:54.907405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 559/1095] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:54.913038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:54.977543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078a439-586e-4acb-9535-33b17f308fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:54.978920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078a439-586e-4acb-9535-33b17f308fc8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:55.009943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:55.012313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946a193-aa1a-4d10-a7df-feaaf086884d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:55.046897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:55.047727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:55.048010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:55.048278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:55.055344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:55.056790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:08:55.070994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:55.072019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:08:55.073014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 560/1096] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:08:55.078046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:55.080911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:55.085807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 537/1097] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:08:55.092159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078a439-586e-4acb-9535-33b17f308fc8 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:55.179127 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe28f6a-386d-434b-951f-899da199def8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:55.189050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078a439-586e-4acb-9535-33b17f308fc8 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:55.213450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:55.214123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:55.214405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:55.214678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:55.215167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:55.215808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:55.215808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:55.217283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:55.224984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa36371-2f7b-4f41-9856-37b6ed6dabbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:55.225858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa36371-2f7b-4f41-9856-37b6ed6dabbb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:55.226244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa36371-2f7b-4f41-9856-37b6ed6dabbb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:55.226529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa36371-2f7b-4f41-9856-37b6ed6dabbb None admin] PATH_INFO: `/v3/projects/f7ff831214604ae4954c77bec590ddc6` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:55.227804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa36371-2f7b-4f41-9856-37b6ed6dabbb None admin] RBAC: Authorizing `identity:get_project(project_id=f7ff831214604ae4954c77bec590ddc6)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:55.229171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa36371-2f7b-4f41-9856-37b6ed6dabbb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:55.231037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 538/1098] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:08:55 2022] GET /identity/v3/projects/f7ff831214604ae4954c77bec590ddc6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:55.235004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 561/1099] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 08:08:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:08:55.237052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa36371-2f7b-4f41-9856-37b6ed6dabbb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:55.240357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:55.297945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:55.339600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa36371-2f7b-4f41-9856-37b6ed6dabbb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:55.340097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953b51a-6245-42f7-903c-7dc4baf16d7e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:55.372802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:55.373174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abfcfa-9649-4bf5-8c55-259499154de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:55.373455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:55.373738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:55.373837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abfcfa-9649-4bf5-8c55-259499154de8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:55.374015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:55.374108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abfcfa-9649-4bf5-8c55-259499154de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:55.374347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abfcfa-9649-4bf5-8c55-259499154de8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:55.374843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=f7ff831214604ae4954c77bec590ddc6, user.id=3b2e25f4b7e04d688beb57e7b92ada26) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:55.375159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:55.376691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:55.383052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 539/1100] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:55 2022] GET /identity/v3/role_assignments?user.id=3b2e25f4b7e04d688beb57e7b92ada26&scope.project.id=f7ff831214604ae4954c77bec590ddc6&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:55.447011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abfcfa-9649-4bf5-8c55-259499154de8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:55.448519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abfcfa-9649-4bf5-8c55-259499154de8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:55.514932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:55.566839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 562/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:08:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 08:08:55.572918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:55.633265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:55.678399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f898f7e-0135-4656-91d5-f40def7ecf18 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:55.712442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:55.713231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:55.713510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:55.713779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:55.724685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:55.726132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:08:55.743652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:55.744620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:55.745713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:08:55.745713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:08:55.745713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:08:55.745713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:55.745713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 540/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:55.750939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abfcfa-9649-4bf5-8c55-259499154de8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:55.850657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abfcfa-9649-4bf5-8c55-259499154de8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:55.885829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:55.886493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:55.886770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:55.887041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:55.887787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:55.888086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:55.889473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:55.901158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 563/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:55.906809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:56.007541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d0a11-23c9-4c95-b36f-62c11d262220 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:56.022440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:56.049309 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:56.050159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:56.050462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:56.050735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:56.057827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:56.059310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:08:56.073759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:08:56.074752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:08:56.076126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 541/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:08:56.080834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:56.103264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:56.182663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac20e7-e6e4-4de1-95c4-d580452a69fe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:56.216979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff22df5-cb96-4bb3-a485-5016bd69b9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:56.217639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff22df5-cb96-4bb3-a485-5016bd69b9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:56.217920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff22df5-cb96-4bb3-a485-5016bd69b9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:56.218191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff22df5-cb96-4bb3-a485-5016bd69b9a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:56.218968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff22df5-cb96-4bb3-a485-5016bd69b9a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:56.219283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff22df5-cb96-4bb3-a485-5016bd69b9a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:56.220744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff22df5-cb96-4bb3-a485-5016bd69b9a2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:56.232766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 564/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:08:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:56.239123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:56.344701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02c9a8-21b3-49fb-b01a-6dbce8c55bbd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:56.381633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4ec3-3de6-459b-89b2-f23b61caf44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:56.382282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4ec3-3de6-459b-89b2-f23b61caf44a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:56.382561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4ec3-3de6-459b-89b2-f23b61caf44a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:56.382793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4ec3-3de6-459b-89b2-f23b61caf44a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:56.383665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4ec3-3de6-459b-89b2-f23b61caf44a None admin] RBAC: Adding query filter params (role.id=52fd1ff24fe24ad98ca611cd5f52669f, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=96456bfed82d469b888bd495ec46e796) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:08:56.383967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4ec3-3de6-459b-89b2-f23b61caf44a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:56.385398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4ec3-3de6-459b-89b2-f23b61caf44a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:56.391555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 542/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:08:56 2022] GET /identity/v3/role_assignments?user.id=96456bfed82d469b888bd495ec46e796&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=52fd1ff24fe24ad98ca611cd5f52669f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:57.438817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe026a7d-a9c5-4345-a6fb-fe906083eec6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:57.439774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe026a7d-a9c5-4345-a6fb-fe906083eec6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:57.439774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe026a7d-a9c5-4345-a6fb-fe906083eec6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:57.441998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 565/1107] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:57.450820 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cb6d9-7307-4f49-bc94-013e2543ff66 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:57.451293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cb6d9-7307-4f49-bc94-013e2543ff66 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:57.451432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cb6d9-7307-4f49-bc94-013e2543ff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:57.470385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6b7cb6d9-7307-4f49-bc94-013e2543ff66 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:57.471631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6b7cb6d9-7307-4f49-bc94-013e2543ff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:57.501696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:57.548411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 543/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:08:57.573567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe026a7d-a9c5-4345-a6fb-fe906083eec6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:57.661726 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:08:57.683244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe026a7d-a9c5-4345-a6fb-fe906083eec6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:57.723738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa040f8b-3a40-4f2d-8234-d22500841540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:57.724460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa040f8b-3a40-4f2d-8234-d22500841540 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:57.724740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa040f8b-3a40-4f2d-8234-d22500841540 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:57.725006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa040f8b-3a40-4f2d-8234-d22500841540 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:57.726196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa040f8b-3a40-4f2d-8234-d22500841540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:57.727697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa040f8b-3a40-4f2d-8234-d22500841540 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:57.729284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 566/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:08:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:08:57.738649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7cb6d9-7307-4f49-bc94-013e2543ff66 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:08:57.847720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7cb6d9-7307-4f49-bc94-013e2543ff66 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:08:57.883426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9565d-7a43-4ff3-8578-ffdd0f78d2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:08:57.884086 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9565d-7a43-4ff3-8578-ffdd0f78d2c4 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:57.884368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9565d-7a43-4ff3-8578-ffdd0f78d2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:57.884647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9565d-7a43-4ff3-8578-ffdd0f78d2c4 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:57.885355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9565d-7a43-4ff3-8578-ffdd0f78d2c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:08:57.886787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9565d-7a43-4ff3-8578-ffdd0f78d2c4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:08:57.908110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 544/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:08:57 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:08:59.944374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95afe2-ed46-41c8-8578-c91b7b8d5ab5 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:59.945318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95afe2-ed46-41c8-8578-c91b7b8d5ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:59.945318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95afe2-ed46-41c8-8578-c91b7b8d5ab5 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:59.947459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 567/1111] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:08:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:08:59.956269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27e467-6c70-42e4-83f7-f6bf01df6d64 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:08:59.956700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27e467-6c70-42e4-83f7-f6bf01df6d64 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:08:59.956844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27e467-6c70-42e4-83f7-f6bf01df6d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:08:59.980270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5d27e467-6c70-42e4-83f7-f6bf01df6d64 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:08:59.981456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5d27e467-6c70-42e4-83f7-f6bf01df6d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:08:59.996610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:00.052622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 545/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:08:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:00.077113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95afe2-ed46-41c8-8578-c91b7b8d5ab5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:00.187803 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95afe2-ed46-41c8-8578-c91b7b8d5ab5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:00.225249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8a99f-6df4-4615-a225-b55d2549b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:00.226035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8a99f-6df4-4615-a225-b55d2549b863 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:00.226318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8a99f-6df4-4615-a225-b55d2549b863 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:00.226593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8a99f-6df4-4615-a225-b55d2549b863 None admin] PATH_INFO: `/v3/roles/798b7bcb53684ab49ae899f8902f444a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:00.228093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8a99f-6df4-4615-a225-b55d2549b863 None admin] RBAC: Authorizing `identity:get_role(role_id=798b7bcb53684ab49ae899f8902f444a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:00.229484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8a99f-6df4-4615-a225-b55d2549b863 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:00.230846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 568/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:00 2022] GET /identity/v3/roles/798b7bcb53684ab49ae899f8902f444a => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:00.237473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27e467-6c70-42e4-83f7-f6bf01df6d64 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:00.336299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27e467-6c70-42e4-83f7-f6bf01df6d64 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:00.372236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2217dc-00de-4cb7-a1ce-8599c7d7253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:00.373104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2217dc-00de-4cb7-a1ce-8599c7d7253a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:00.373383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2217dc-00de-4cb7-a1ce-8599c7d7253a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:00.373656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2217dc-00de-4cb7-a1ce-8599c7d7253a None admin] PATH_INFO: `/v3/users/9ad2cd66683c4df5a745eb7fe809bed2` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:00.387435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2217dc-00de-4cb7-a1ce-8599c7d7253a None admin] RBAC: Authorizing `identity:get_user(user_id=9ad2cd66683c4df5a745eb7fe809bed2)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:00.388885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2217dc-00de-4cb7-a1ce-8599c7d7253a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:00.390529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 546/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:00 2022] GET /identity/v3/users/9ad2cd66683c4df5a745eb7fe809bed2 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:00.396837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8a99f-6df4-4615-a225-b55d2549b863 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:00.416524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:00.508804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8a99f-6df4-4615-a225-b55d2549b863 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:00.549399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827333b-41f0-4a8d-b634-1dd4b929155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:00.550040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827333b-41f0-4a8d-b634-1dd4b929155d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:00.550318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827333b-41f0-4a8d-b634-1dd4b929155d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:00.550590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827333b-41f0-4a8d-b634-1dd4b929155d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:00.619925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827333b-41f0-4a8d-b634-1dd4b929155d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:00.621696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827333b-41f0-4a8d-b634-1dd4b929155d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:00.638457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:00.727414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 569/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:09:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:09:00.733564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2217dc-00de-4cb7-a1ce-8599c7d7253a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:00.744713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:00.847614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2217dc-00de-4cb7-a1ce-8599c7d7253a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:00.882761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba94d8-fb7d-4ced-a921-412177f817be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:00.883636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba94d8-fb7d-4ced-a921-412177f817be None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:00.883913 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba94d8-fb7d-4ced-a921-412177f817be None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:00.884181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba94d8-fb7d-4ced-a921-412177f817be None admin] PATH_INFO: `/v3/projects/f7ff831214604ae4954c77bec590ddc6` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:00.885465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba94d8-fb7d-4ced-a921-412177f817be None admin] RBAC: Authorizing `identity:get_project(project_id=f7ff831214604ae4954c77bec590ddc6)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:00.886904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba94d8-fb7d-4ced-a921-412177f817be None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:00.888614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 547/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:00 2022] GET /identity/v3/projects/f7ff831214604ae4954c77bec590ddc6 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:09:00.895592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827333b-41f0-4a8d-b634-1dd4b929155d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:01.004091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827333b-41f0-4a8d-b634-1dd4b929155d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:01.020682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:01.040376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907a7de-0771-4fca-837b-649e309366f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:01.041016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907a7de-0771-4fca-837b-649e309366f5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:01.041294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907a7de-0771-4fca-837b-649e309366f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:01.041562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907a7de-0771-4fca-837b-649e309366f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:01.042384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907a7de-0771-4fca-837b-649e309366f5 None admin] RBAC: Adding query filter params (role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=f7ff831214604ae4954c77bec590ddc6, user.id=9ad2cd66683c4df5a745eb7fe809bed2) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:01.042687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907a7de-0771-4fca-837b-649e309366f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:01.044156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907a7de-0771-4fca-837b-649e309366f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:01.049743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 570/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:09:00 2022] GET /identity/v3/role_assignments?user.id=9ad2cd66683c4df5a745eb7fe809bed2&scope.project.id=f7ff831214604ae4954c77bec590ddc6&role.id=798b7bcb53684ab49ae899f8902f444a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:03.026192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ecec-22e5-4147-8a52-dc70b15c1b26 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:03.026786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ecec-22e5-4147-8a52-dc70b15c1b26 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:03.026786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ecec-22e5-4147-8a52-dc70b15c1b26 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:03.028148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 548/1118] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:03.034416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359ee90-e4a5-48bf-8f00-c8d01efd7247 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:03.035180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359ee90-e4a5-48bf-8f00-c8d01efd7247 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:03.035368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359ee90-e4a5-48bf-8f00-c8d01efd7247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:03.060004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e359ee90-e4a5-48bf-8f00-c8d01efd7247 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:03.061891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e359ee90-e4a5-48bf-8f00-c8d01efd7247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:03.138988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 571/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:03.161927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ecec-22e5-4147-8a52-dc70b15c1b26 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:03.247328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:03.275564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ecec-22e5-4147-8a52-dc70b15c1b26 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:03.311037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca2b1a-d6ab-4349-895f-0ef8c5c0f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:03.311855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca2b1a-d6ab-4349-895f-0ef8c5c0f72e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:03.312126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca2b1a-d6ab-4349-895f-0ef8c5c0f72e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:03.312399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca2b1a-d6ab-4349-895f-0ef8c5c0f72e None admin] PATH_INFO: `/v3/roles/798b7bcb53684ab49ae899f8902f444a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:03.313976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca2b1a-d6ab-4349-895f-0ef8c5c0f72e None admin] RBAC: Authorizing `identity:get_role(role_id=798b7bcb53684ab49ae899f8902f444a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:03.315539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca2b1a-d6ab-4349-895f-0ef8c5c0f72e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:03.316893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 549/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:03 2022] GET /identity/v3/roles/798b7bcb53684ab49ae899f8902f444a => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:03.323759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359ee90-e4a5-48bf-8f00-c8d01efd7247 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:03.436439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359ee90-e4a5-48bf-8f00-c8d01efd7247 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:03.447194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:03.472789 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7bd9b-2276-48a1-a60f-02412b9b45de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:03.473681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7bd9b-2276-48a1-a60f-02412b9b45de None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:03.473960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7bd9b-2276-48a1-a60f-02412b9b45de None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:03.474231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7bd9b-2276-48a1-a60f-02412b9b45de None admin] PATH_INFO: `/v3/users/9ad2cd66683c4df5a745eb7fe809bed2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:03.475603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7bd9b-2276-48a1-a60f-02412b9b45de None admin] RBAC: Authorizing `identity:get_user(user_id=9ad2cd66683c4df5a745eb7fe809bed2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:03.477075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7bd9b-2276-48a1-a60f-02412b9b45de None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:03.478735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 572/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:03 2022] GET /identity/v3/users/9ad2cd66683c4df5a745eb7fe809bed2 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:03.484945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca2b1a-d6ab-4349-895f-0ef8c5c0f72e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:03.589830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca2b1a-d6ab-4349-895f-0ef8c5c0f72e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:03.626426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952f69b-1846-453d-b282-d8a93dbb5792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:03.627475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952f69b-1846-453d-b282-d8a93dbb5792 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:03.627866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952f69b-1846-453d-b282-d8a93dbb5792 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:03.628067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952f69b-1846-453d-b282-d8a93dbb5792 None admin] PATH_INFO: `/v3/projects/f7ff831214604ae4954c77bec590ddc6` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:03.629283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952f69b-1846-453d-b282-d8a93dbb5792 None admin] RBAC: Authorizing `identity:get_project(project_id=f7ff831214604ae4954c77bec590ddc6)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:03.630722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952f69b-1846-453d-b282-d8a93dbb5792 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:03.632439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 550/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:03 2022] GET /identity/v3/projects/f7ff831214604ae4954c77bec590ddc6 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:03.638550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7bd9b-2276-48a1-a60f-02412b9b45de None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:03.754804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7bd9b-2276-48a1-a60f-02412b9b45de None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:03.794104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c741cd-1189-4be2-9212-f07488848d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:03.795022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c741cd-1189-4be2-9212-f07488848d08 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:03.795440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c741cd-1189-4be2-9212-f07488848d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:03.795578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c741cd-1189-4be2-9212-f07488848d08 None admin] PATH_INFO: `/v3/projects/f7ff831214604ae4954c77bec590ddc6/users/9ad2cd66683c4df5a745eb7fe809bed2/roles/798b7bcb53684ab49ae899f8902f444a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:03.798208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c741cd-1189-4be2-9212-f07488848d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7ff831214604ae4954c77bec590ddc6, user_id=9ad2cd66683c4df5a745eb7fe809bed2, role_id=798b7bcb53684ab49ae899f8902f444a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:03.799729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c741cd-1189-4be2-9212-f07488848d08 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:03.816295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 573/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:09:03 2022] PUT /identity/v3/projects/f7ff831214604ae4954c77bec590ddc6/users/9ad2cd66683c4df5a745eb7fe809bed2/roles/798b7bcb53684ab49ae899f8902f444a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:09:05.794358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7020450-eb51-47ae-bc35-ac64099f4311 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:05.794912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7020450-eb51-47ae-bc35-ac64099f4311 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:05.794912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7020450-eb51-47ae-bc35-ac64099f4311 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:05.796241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 551/1124] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:05.802608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa819-6abf-44fb-a91f-a65779ad2aaf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:05.803310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa819-6abf-44fb-a91f-a65779ad2aaf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:05.803464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa819-6abf-44fb-a91f-a65779ad2aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:05.825299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-451aa819-6abf-44fb-a91f-a65779ad2aaf None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:05.826469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-451aa819-6abf-44fb-a91f-a65779ad2aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:05.857459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:05.897176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 574/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:05.921401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7020450-eb51-47ae-bc35-ac64099f4311 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:05.938989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:06.044745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7020450-eb51-47ae-bc35-ac64099f4311 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:06.083649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fae5f-6dfe-4275-85bb-e4f3d491c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:06.084506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae5f-6dfe-4275-85bb-e4f3d491c224 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:06.084784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae5f-6dfe-4275-85bb-e4f3d491c224 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:06.085055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae5f-6dfe-4275-85bb-e4f3d491c224 None admin] PATH_INFO: `/v3/roles/798b7bcb53684ab49ae899f8902f444a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:06.086575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fae5f-6dfe-4275-85bb-e4f3d491c224 None admin] RBAC: Authorizing `identity:get_role(role_id=798b7bcb53684ab49ae899f8902f444a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:06.087977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fae5f-6dfe-4275-85bb-e4f3d491c224 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:06.089332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 552/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:05 2022] GET /identity/v3/roles/798b7bcb53684ab49ae899f8902f444a => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:06.095233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa819-6abf-44fb-a91f-a65779ad2aaf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:06.217020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa819-6abf-44fb-a91f-a65779ad2aaf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:06.253943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c6952-4783-40c7-a7e9-f0695165ea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:06.254806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c6952-4783-40c7-a7e9-f0695165ea5b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:06.255084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c6952-4783-40c7-a7e9-f0695165ea5b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:06.255394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c6952-4783-40c7-a7e9-f0695165ea5b None admin] PATH_INFO: `/v3/users/9ad2cd66683c4df5a745eb7fe809bed2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:06.256691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c6952-4783-40c7-a7e9-f0695165ea5b None admin] RBAC: Authorizing `identity:get_user(user_id=9ad2cd66683c4df5a745eb7fe809bed2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:06.258139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c6952-4783-40c7-a7e9-f0695165ea5b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:06.259799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 575/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:06 2022] GET /identity/v3/users/9ad2cd66683c4df5a745eb7fe809bed2 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:09:06.266464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fae5f-6dfe-4275-85bb-e4f3d491c224 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:06.354506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:06.378078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fae5f-6dfe-4275-85bb-e4f3d491c224 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:06.414622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bb329-8d3e-4fd5-a459-1e5ae2ff58f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:06.415297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bb329-8d3e-4fd5-a459-1e5ae2ff58f5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:06.415578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bb329-8d3e-4fd5-a459-1e5ae2ff58f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:06.415846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bb329-8d3e-4fd5-a459-1e5ae2ff58f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:06.485288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bb329-8d3e-4fd5-a459-1e5ae2ff58f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:06.486857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bb329-8d3e-4fd5-a459-1e5ae2ff58f5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:06.576205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:06.599560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 553/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:09:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:09:06.605907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c6952-4783-40c7-a7e9-f0695165ea5b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:06.645874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:06.711741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c6952-4783-40c7-a7e9-f0695165ea5b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:06.753102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d980941-ebda-41e0-9d01-f4bd31883847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:06.753980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d980941-ebda-41e0-9d01-f4bd31883847 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:06.754262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d980941-ebda-41e0-9d01-f4bd31883847 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:06.754536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d980941-ebda-41e0-9d01-f4bd31883847 None admin] PATH_INFO: `/v3/projects/f7ff831214604ae4954c77bec590ddc6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:06.755832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d980941-ebda-41e0-9d01-f4bd31883847 None admin] RBAC: Authorizing `identity:get_project(project_id=f7ff831214604ae4954c77bec590ddc6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:06.757300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d980941-ebda-41e0-9d01-f4bd31883847 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:06.759103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 576/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:06 2022] GET /identity/v3/projects/f7ff831214604ae4954c77bec590ddc6 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:06.766152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bb329-8d3e-4fd5-a459-1e5ae2ff58f5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:06.876484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bb329-8d3e-4fd5-a459-1e5ae2ff58f5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:06.912407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7506ca-f5e2-4cd9-946c-bf48080f5a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:06.913044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7506ca-f5e2-4cd9-946c-bf48080f5a83 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:06.913321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7506ca-f5e2-4cd9-946c-bf48080f5a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:06.913592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7506ca-f5e2-4cd9-946c-bf48080f5a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:06.914419 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7506ca-f5e2-4cd9-946c-bf48080f5a83 None admin] RBAC: Adding query filter params (role.id=798b7bcb53684ab49ae899f8902f444a, scope.project.id=f7ff831214604ae4954c77bec590ddc6, user.id=9ad2cd66683c4df5a745eb7fe809bed2) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:06.914729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7506ca-f5e2-4cd9-946c-bf48080f5a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:06.916194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7506ca-f5e2-4cd9-946c-bf48080f5a83 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:06.922076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 554/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:09:06 2022] GET /identity/v3/role_assignments?user.id=9ad2cd66683c4df5a745eb7fe809bed2&scope.project.id=f7ff831214604ae4954c77bec590ddc6&role.id=798b7bcb53684ab49ae899f8902f444a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:08.937712 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17df05-4f79-4ff7-a33c-4a7a2ad258a2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:08.938272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17df05-4f79-4ff7-a33c-4a7a2ad258a2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:08.938272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17df05-4f79-4ff7-a33c-4a7a2ad258a2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:08.939755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 577/1131] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:08.945935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81fcfb-52c6-407b-8c56-8517aaee9052 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:08.946704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81fcfb-52c6-407b-8c56-8517aaee9052 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:08.946835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81fcfb-52c6-407b-8c56-8517aaee9052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:08.973509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1e81fcfb-52c6-407b-8c56-8517aaee9052 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:08.974679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1e81fcfb-52c6-407b-8c56-8517aaee9052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:09.019348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:09.047369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 555/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:09.070591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17df05-4f79-4ff7-a33c-4a7a2ad258a2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:09.176728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17df05-4f79-4ff7-a33c-4a7a2ad258a2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:09.179009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:09.222776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4adb5-9026-41ef-aee4-c277861078b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:09.223534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4adb5-9026-41ef-aee4-c277861078b3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:09.223820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4adb5-9026-41ef-aee4-c277861078b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:09.224096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4adb5-9026-41ef-aee4-c277861078b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:09.225320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4adb5-9026-41ef-aee4-c277861078b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:09.226825 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4adb5-9026-41ef-aee4-c277861078b3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:09.228524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 578/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:09:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:09.237243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81fcfb-52c6-407b-8c56-8517aaee9052 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:09.358703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81fcfb-52c6-407b-8c56-8517aaee9052 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:09.395800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1382fd-9111-474a-b2f6-0af96768e2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:09.396454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1382fd-9111-474a-b2f6-0af96768e2d5 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:09.396730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1382fd-9111-474a-b2f6-0af96768e2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:09.397003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1382fd-9111-474a-b2f6-0af96768e2d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:09.397720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1382fd-9111-474a-b2f6-0af96768e2d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:09.399176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1382fd-9111-474a-b2f6-0af96768e2d5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:09.428677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 556/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:09:09 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:09:11.457757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c57c-0a8b-4eff-8142-ad7d7274606f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:11.458668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c57c-0a8b-4eff-8142-ad7d7274606f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:11.458668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c57c-0a8b-4eff-8142-ad7d7274606f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:11.460920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 579/1135] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:11.469513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb38d8-54aa-4ba0-86e7-6be19daf0010 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:11.469952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb38d8-54aa-4ba0-86e7-6be19daf0010 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:11.470088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb38d8-54aa-4ba0-86e7-6be19daf0010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:11.492614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-81bb38d8-54aa-4ba0-86e7-6be19daf0010 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:11.493806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-81bb38d8-54aa-4ba0-86e7-6be19daf0010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:11.516015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:11.565207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 557/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:11.597117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1c57c-0a8b-4eff-8142-ad7d7274606f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:11.710700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1c57c-0a8b-4eff-8142-ad7d7274606f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:11.747155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131c2af-5647-430c-8b3e-242d135d0f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:11.747978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c2af-5647-430c-8b3e-242d135d0f65 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:11.748264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c2af-5647-430c-8b3e-242d135d0f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:11.748535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c2af-5647-430c-8b3e-242d135d0f65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:11.749826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131c2af-5647-430c-8b3e-242d135d0f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:11.751306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131c2af-5647-430c-8b3e-242d135d0f65 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:11.753430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 580/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:09:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:11.759372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb38d8-54aa-4ba0-86e7-6be19daf0010 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:11.880585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb38d8-54aa-4ba0-86e7-6be19daf0010 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:11.917182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e713990-36dc-4964-a54b-73451ffa6956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:11.917854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e713990-36dc-4964-a54b-73451ffa6956 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:11.918137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e713990-36dc-4964-a54b-73451ffa6956 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:11.918408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e713990-36dc-4964-a54b-73451ffa6956 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:11.919120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e713990-36dc-4964-a54b-73451ffa6956 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:11.920602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e713990-36dc-4964-a54b-73451ffa6956 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:11.946085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 558/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:09:11 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:09:13.964676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf709-abcf-4cd4-9800-072ea53af398 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:13.965266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf709-abcf-4cd4-9800-072ea53af398 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:13.965266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf709-abcf-4cd4-9800-072ea53af398 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:13.966583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 581/1139] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:13.973020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfbd8-49b7-41f9-95c7-43aaff6dddfb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:13.973754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfbd8-49b7-41f9-95c7-43aaff6dddfb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:13.973901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfbd8-49b7-41f9-95c7-43aaff6dddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:13.996667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-abcdfbd8-49b7-41f9-95c7-43aaff6dddfb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:13.997858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-abcdfbd8-49b7-41f9-95c7-43aaff6dddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:14.005663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:14.074182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 559/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:14.106544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf709-abcf-4cd4-9800-072ea53af398 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:14.152541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:14.215743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf709-abcf-4cd4-9800-072ea53af398 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:14.250011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40a966-de17-4cf0-aaf0-5ccf6717fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:14.250796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40a966-de17-4cf0-aaf0-5ccf6717fad9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:14.251075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40a966-de17-4cf0-aaf0-5ccf6717fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:14.251378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40a966-de17-4cf0-aaf0-5ccf6717fad9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:14.261635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40a966-de17-4cf0-aaf0-5ccf6717fad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1a40a966-de17-4cf0-aaf0-5ccf6717fad9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:09:14.269245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:09:14.270361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:09:14.271490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 582/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:09:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:09:14.279094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdfbd8-49b7-41f9-95c7-43aaff6dddfb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:14.386466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdfbd8-49b7-41f9-95c7-43aaff6dddfb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:14.422352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:14.423008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:14.423310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:14.423653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:14.424501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:14.424806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:14.426178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:14.434491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 560/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:09:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:14.443020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40a966-de17-4cf0-aaf0-5ccf6717fad9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:14.557490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:14.562077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40a966-de17-4cf0-aaf0-5ccf6717fad9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:14.606336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0be848-f8ee-4fcc-a6c1-5d98d02f2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:14.607281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0be848-f8ee-4fcc-a6c1-5d98d02f2d35 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:14.607605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0be848-f8ee-4fcc-a6c1-5d98d02f2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:14.607880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0be848-f8ee-4fcc-a6c1-5d98d02f2d35 None admin] PATH_INFO: `/v3/users/5e690433eda74a839ec4039b864842dd` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:14.622147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0be848-f8ee-4fcc-a6c1-5d98d02f2d35 None admin] RBAC: Authorizing `identity:get_user(user_id=5e690433eda74a839ec4039b864842dd)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:14.623642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0be848-f8ee-4fcc-a6c1-5d98d02f2d35 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:14.625305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 583/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:14 2022] GET /identity/v3/users/5e690433eda74a839ec4039b864842dd => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:14.631853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:14.636229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:14.749196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6d2f1-20ab-4edb-89df-3fcf07e524d6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:14.785093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e1531-a159-4f54-9eec-b50df7159a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:14.785667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e1531-a159-4f54-9eec-b50df7159a93 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:14.785952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e1531-a159-4f54-9eec-b50df7159a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:14.786225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e1531-a159-4f54-9eec-b50df7159a93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:14.855078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e1531-a159-4f54-9eec-b50df7159a93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:14.856550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e1531-a159-4f54-9eec-b50df7159a93 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:14.859155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:14.960609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 561/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:09:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:09:14.970271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0be848-f8ee-4fcc-a6c1-5d98d02f2d35 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:15.079098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0be848-f8ee-4fcc-a6c1-5d98d02f2d35 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:15.114078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d3af6-f7ae-42b3-9446-620763112e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:15.114928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d3af6-f7ae-42b3-9446-620763112e48 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:15.115242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d3af6-f7ae-42b3-9446-620763112e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:15.115504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d3af6-f7ae-42b3-9446-620763112e48 None admin] PATH_INFO: `/v3/projects/fb94a5f7be4a4a0a9fc48ee1ec4ad130` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:15.116690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d3af6-f7ae-42b3-9446-620763112e48 None admin] RBAC: Authorizing `identity:get_project(project_id=fb94a5f7be4a4a0a9fc48ee1ec4ad130)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:15.118127 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d3af6-f7ae-42b3-9446-620763112e48 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:15.119844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 584/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:14 2022] GET /identity/v3/projects/fb94a5f7be4a4a0a9fc48ee1ec4ad130 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:15.126737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e1531-a159-4f54-9eec-b50df7159a93 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:15.238486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e1531-a159-4f54-9eec-b50df7159a93 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:15.240908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:15.285088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb230bf-3782-4c5a-8ae9-3e28efb230f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:15.285749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb230bf-3782-4c5a-8ae9-3e28efb230f3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:15.286034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb230bf-3782-4c5a-8ae9-3e28efb230f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:15.286311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb230bf-3782-4c5a-8ae9-3e28efb230f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:15.287146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb230bf-3782-4c5a-8ae9-3e28efb230f3 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=fb94a5f7be4a4a0a9fc48ee1ec4ad130, user.id=5e690433eda74a839ec4039b864842dd) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:15.287488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb230bf-3782-4c5a-8ae9-3e28efb230f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:15.288968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb230bf-3782-4c5a-8ae9-3e28efb230f3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:15.295387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 562/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:09:15 2022] GET /identity/v3/role_assignments?user.id=5e690433eda74a839ec4039b864842dd&scope.project.id=fb94a5f7be4a4a0a9fc48ee1ec4ad130&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:17.284285 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e897836-2a51-43a0-9b80-001ac3aa7ddc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:17.284889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e897836-2a51-43a0-9b80-001ac3aa7ddc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:17.284889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e897836-2a51-43a0-9b80-001ac3aa7ddc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:17.286192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 585/1147] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:17.292738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12303-cfa3-4b85-b171-b44c2ccc779f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:17.293485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12303-cfa3-4b85-b171-b44c2ccc779f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:17.293661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12303-cfa3-4b85-b171-b44c2ccc779f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:17.319591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-21a12303-cfa3-4b85-b171-b44c2ccc779f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:17.320774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-21a12303-cfa3-4b85-b171-b44c2ccc779f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:17.400314 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 563/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:17.430611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e897836-2a51-43a0-9b80-001ac3aa7ddc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:17.460677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:17.551613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e897836-2a51-43a0-9b80-001ac3aa7ddc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:17.587200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc596d3-7578-41b6-a113-ed413fcb49f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:17.588017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc596d3-7578-41b6-a113-ed413fcb49f4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:17.588439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc596d3-7578-41b6-a113-ed413fcb49f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:17.588717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc596d3-7578-41b6-a113-ed413fcb49f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:17.599347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc596d3-7578-41b6-a113-ed413fcb49f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8cc596d3-7578-41b6-a113-ed413fcb49f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:09:17.606878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:09:17.607900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:09:17.608943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 586/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:09:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:09:17.614748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12303-cfa3-4b85-b171-b44c2ccc779f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:17.716761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:17.721088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12303-cfa3-4b85-b171-b44c2ccc779f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:17.757826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:17.758483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:17.758764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:17.759040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:17.759801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:17.760106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:17.761487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:17.770995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 564/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:09:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:17.780426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc596d3-7578-41b6-a113-ed413fcb49f4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:17.874424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:17.889849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc596d3-7578-41b6-a113-ed413fcb49f4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:17.924674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d19e-a247-45f5-83ed-bac3f7f2f224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:17.925541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d19e-a247-45f5-83ed-bac3f7f2f224 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:17.925790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d19e-a247-45f5-83ed-bac3f7f2f224 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:17.926087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d19e-a247-45f5-83ed-bac3f7f2f224 None admin] PATH_INFO: `/v3/users/5e690433eda74a839ec4039b864842dd` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:17.927413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066d19e-a247-45f5-83ed-bac3f7f2f224 None admin] RBAC: Authorizing `identity:get_user(user_id=5e690433eda74a839ec4039b864842dd)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:17.928854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066d19e-a247-45f5-83ed-bac3f7f2f224 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:17.930479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 587/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:17 2022] GET /identity/v3/users/5e690433eda74a839ec4039b864842dd => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:17.936762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:18.057165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0be38-360d-4782-b87c-cd84fde87d54 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:18.091347 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb65ed-1bda-49e0-86cb-8c87c6902553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:18.092192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb65ed-1bda-49e0-86cb-8c87c6902553 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:18.092462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb65ed-1bda-49e0-86cb-8c87c6902553 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:18.092726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb65ed-1bda-49e0-86cb-8c87c6902553 None admin] PATH_INFO: `/v3/projects/fb94a5f7be4a4a0a9fc48ee1ec4ad130` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:18.093936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb65ed-1bda-49e0-86cb-8c87c6902553 None admin] RBAC: Authorizing `identity:get_project(project_id=fb94a5f7be4a4a0a9fc48ee1ec4ad130)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:18.095444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb65ed-1bda-49e0-86cb-8c87c6902553 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:18.097095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 565/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:17 2022] GET /identity/v3/projects/fb94a5f7be4a4a0a9fc48ee1ec4ad130 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:18.103820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d19e-a247-45f5-83ed-bac3f7f2f224 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:18.219076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d19e-a247-45f5-83ed-bac3f7f2f224 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:18.262248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0d35a-8f84-4735-9533-6e027d2e5019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:18.263130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d35a-8f84-4735-9533-6e027d2e5019 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:18.263446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d35a-8f84-4735-9533-6e027d2e5019 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:18.263719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d35a-8f84-4735-9533-6e027d2e5019 None admin] PATH_INFO: `/v3/projects/fb94a5f7be4a4a0a9fc48ee1ec4ad130/users/5e690433eda74a839ec4039b864842dd/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:18.265946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0d35a-8f84-4735-9533-6e027d2e5019 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb94a5f7be4a4a0a9fc48ee1ec4ad130, user_id=5e690433eda74a839ec4039b864842dd, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:18.267576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0d35a-8f84-4735-9533-6e027d2e5019 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:18.283531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 588/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:09:18 2022] PUT /identity/v3/projects/fb94a5f7be4a4a0a9fc48ee1ec4ad130/users/5e690433eda74a839ec4039b864842dd/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:09:20.266368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa7dc2-61bb-46de-8ea1-58b6299c32e9 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:20.266961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa7dc2-61bb-46de-8ea1-58b6299c32e9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:20.266961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa7dc2-61bb-46de-8ea1-58b6299c32e9 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:20.268337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 566/1154] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:20.274908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecf89-f97e-4afd-8dda-4b64f56075bd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:20.275719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecf89-f97e-4afd-8dda-4b64f56075bd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:20.275867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecf89-f97e-4afd-8dda-4b64f56075bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:20.296543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3abecf89-f97e-4afd-8dda-4b64f56075bd None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:20.297728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3abecf89-f97e-4afd-8dda-4b64f56075bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:20.307456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:20.373067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 589/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:20.396744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa7dc2-61bb-46de-8ea1-58b6299c32e9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:20.428505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:20.510187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa7dc2-61bb-46de-8ea1-58b6299c32e9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:20.545329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aca4cb-271c-4e57-88bd-8b408ff1ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:20.546058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aca4cb-271c-4e57-88bd-8b408ff1ff30 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:20.546361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aca4cb-271c-4e57-88bd-8b408ff1ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:20.546632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aca4cb-271c-4e57-88bd-8b408ff1ff30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:20.564135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aca4cb-271c-4e57-88bd-8b408ff1ff30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-47aca4cb-271c-4e57-88bd-8b408ff1ff30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:09:20.576792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:09:20.577877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:09:20.579161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 567/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:09:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:09:20.584427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abecf89-f97e-4afd-8dda-4b64f56075bd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:20.688961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abecf89-f97e-4afd-8dda-4b64f56075bd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:20.727123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:20.727820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:20.728098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:20.728372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:20.729120 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:20.729426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:20.730821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:20.739690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 590/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:09:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:20.745946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aca4cb-271c-4e57-88bd-8b408ff1ff30 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:20.848698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:20.861281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aca4cb-271c-4e57-88bd-8b408ff1ff30 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:20.896292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96373d-5aac-4b2f-ba00-1a5fdff06ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:20.897167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96373d-5aac-4b2f-ba00-1a5fdff06ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:20.897443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96373d-5aac-4b2f-ba00-1a5fdff06ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:20.897715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96373d-5aac-4b2f-ba00-1a5fdff06ba7 None admin] PATH_INFO: `/v3/users/5e690433eda74a839ec4039b864842dd` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:20.898941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96373d-5aac-4b2f-ba00-1a5fdff06ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=5e690433eda74a839ec4039b864842dd)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:20.900431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96373d-5aac-4b2f-ba00-1a5fdff06ba7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:20.902043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 568/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:20 2022] GET /identity/v3/users/5e690433eda74a839ec4039b864842dd => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:20.908137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:20.922617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:21.028540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6d3d-4608-4921-81ad-d43f04aa17b0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:21.063120 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8cb95-9cab-44f7-b062-baff3b111057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:21.063779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8cb95-9cab-44f7-b062-baff3b111057 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:21.064138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8cb95-9cab-44f7-b062-baff3b111057 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:21.064408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8cb95-9cab-44f7-b062-baff3b111057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:21.133312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8cb95-9cab-44f7-b062-baff3b111057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:21.134756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8cb95-9cab-44f7-b062-baff3b111057 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:21.142426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:21.238728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 591/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:09:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:09:21.248595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96373d-5aac-4b2f-ba00-1a5fdff06ba7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:21.363614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96373d-5aac-4b2f-ba00-1a5fdff06ba7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:21.399113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec77587-31d8-4152-8eab-6aa94f57f19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:21.399940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec77587-31d8-4152-8eab-6aa94f57f19a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:21.400221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec77587-31d8-4152-8eab-6aa94f57f19a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:21.400491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec77587-31d8-4152-8eab-6aa94f57f19a None admin] PATH_INFO: `/v3/projects/fb94a5f7be4a4a0a9fc48ee1ec4ad130` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:21.401706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec77587-31d8-4152-8eab-6aa94f57f19a None admin] RBAC: Authorizing `identity:get_project(project_id=fb94a5f7be4a4a0a9fc48ee1ec4ad130)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:21.403126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec77587-31d8-4152-8eab-6aa94f57f19a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:21.404863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 569/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:21 2022] GET /identity/v3/projects/fb94a5f7be4a4a0a9fc48ee1ec4ad130 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:21.411986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8cb95-9cab-44f7-b062-baff3b111057 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:21.532604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8cb95-9cab-44f7-b062-baff3b111057 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:21.540852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:21.574646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d1035-acd1-4118-bb7e-5bf6eb89ec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:21.575629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1035-acd1-4118-bb7e-5bf6eb89ec75 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:21.576053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1035-acd1-4118-bb7e-5bf6eb89ec75 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:21.576460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1035-acd1-4118-bb7e-5bf6eb89ec75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:21.577761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d1035-acd1-4118-bb7e-5bf6eb89ec75 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=fb94a5f7be4a4a0a9fc48ee1ec4ad130, user.id=5e690433eda74a839ec4039b864842dd) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:21.578234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d1035-acd1-4118-bb7e-5bf6eb89ec75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:21.580565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d1035-acd1-4118-bb7e-5bf6eb89ec75 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:21.589621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 592/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:09:21 2022] GET /identity/v3/role_assignments?user.id=5e690433eda74a839ec4039b864842dd&scope.project.id=fb94a5f7be4a4a0a9fc48ee1ec4ad130&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:23.601611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d9a1-2ad2-4f17-893c-cea19cc2cc64 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:23.602178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d9a1-2ad2-4f17-893c-cea19cc2cc64 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:23.602178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d9a1-2ad2-4f17-893c-cea19cc2cc64 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:23.603559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 570/1162] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:23.609906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c2027-680d-43ef-b9f4-8dac3d6d1499 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:23.610667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c2027-680d-43ef-b9f4-8dac3d6d1499 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:23.610910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c2027-680d-43ef-b9f4-8dac3d6d1499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:23.629884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9c4c2027-680d-43ef-b9f4-8dac3d6d1499 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:23.631062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9c4c2027-680d-43ef-b9f4-8dac3d6d1499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:23.703703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 593/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:23.726176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d9a1-2ad2-4f17-893c-cea19cc2cc64 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:23.757038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:23.839146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d9a1-2ad2-4f17-893c-cea19cc2cc64 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:23.874029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23b20b-bbed-4353-a1e1-5030dae8f0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:23.874670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23b20b-bbed-4353-a1e1-5030dae8f0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:23.874948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23b20b-bbed-4353-a1e1-5030dae8f0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:23.875246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23b20b-bbed-4353-a1e1-5030dae8f0d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:23.943077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23b20b-bbed-4353-a1e1-5030dae8f0d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:23.944552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23b20b-bbed-4353-a1e1-5030dae8f0d6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:23.968142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:24.043846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 571/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:09:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:09:24.050571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c2027-680d-43ef-b9f4-8dac3d6d1499 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:24.152250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:24.155707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c2027-680d-43ef-b9f4-8dac3d6d1499 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:24.192469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37582-ca63-4f33-9d27-3fd92b2a40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:24.193270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37582-ca63-4f33-9d27-3fd92b2a40f2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:24.193548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37582-ca63-4f33-9d27-3fd92b2a40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:24.193828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37582-ca63-4f33-9d27-3fd92b2a40f2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:24.200736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b37582-ca63-4f33-9d27-3fd92b2a40f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:24.202201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b37582-ca63-4f33-9d27-3fd92b2a40f2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-49b37582-ca63-4f33-9d27-3fd92b2a40f2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:09:24.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:09:24.218022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:09:24.219385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 594/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:09:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:09:24.225036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23b20b-bbed-4353-a1e1-5030dae8f0d6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:24.336371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23b20b-bbed-4353-a1e1-5030dae8f0d6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:24.370048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:24.390266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f147753-1c28-4b92-90cb-d3b923ccb146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:24.391270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f147753-1c28-4b92-90cb-d3b923ccb146 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:24.391662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f147753-1c28-4b92-90cb-d3b923ccb146 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:24.392031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f147753-1c28-4b92-90cb-d3b923ccb146 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:24.392764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f147753-1c28-4b92-90cb-d3b923ccb146 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:24.393072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f147753-1c28-4b92-90cb-d3b923ccb146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:24.394471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f147753-1c28-4b92-90cb-d3b923ccb146 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:24.401995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 572/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 08:09:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:26.439567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66fa3-209f-464a-aca6-f69621da85db None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:26.440440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66fa3-209f-464a-aca6-f69621da85db None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:26.440440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66fa3-209f-464a-aca6-f69621da85db None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:26.442642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 595/1167] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:26.449288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3f78f-49f4-46c3-9488-e13d1723799c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:26.449759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3f78f-49f4-46c3-9488-e13d1723799c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:26.449862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3f78f-49f4-46c3-9488-e13d1723799c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:26.469761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-84b3f78f-49f4-46c3-9488-e13d1723799c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:26.471054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-84b3f78f-49f4-46c3-9488-e13d1723799c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:26.542510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 573/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:26.565852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d66fa3-209f-464a-aca6-f69621da85db None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:26.691424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d66fa3-209f-464a-aca6-f69621da85db None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:26.726728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c5037-d490-4a4a-9052-db9612d35afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:26.727403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c5037-d490-4a4a-9052-db9612d35afe None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:26.727684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c5037-d490-4a4a-9052-db9612d35afe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:26.727959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c5037-d490-4a4a-9052-db9612d35afe None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:26.728695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c5037-d490-4a4a-9052-db9612d35afe None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:26.730086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c5037-d490-4a4a-9052-db9612d35afe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:26.748316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 596/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Thu Jun 23 08:09:26 2022] POST /identity/v3/domains => generated 267 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:09:28.800269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c9a10-f4ce-4c26-8c0c-63061254efcf None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:28.800785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c9a10-f4ce-4c26-8c0c-63061254efcf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:28.800785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c9a10-f4ce-4c26-8c0c-63061254efcf None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:28.802772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 574/1170] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:28.810699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5ebb3-b44a-4bd5-92f2-3cf96f4b328c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:28.811152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5ebb3-b44a-4bd5-92f2-3cf96f4b328c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:28.811284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5ebb3-b44a-4bd5-92f2-3cf96f4b328c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:28.829682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f3c5ebb3-b44a-4bd5-92f2-3cf96f4b328c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:28.830861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f3c5ebb3-b44a-4bd5-92f2-3cf96f4b328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:28.844977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:28.905374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 597/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:09:28.936330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c9a10-f4ce-4c26-8c0c-63061254efcf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:29.053394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c9a10-f4ce-4c26-8c0c-63061254efcf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:29.070616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:29.089198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf3f9b-f743-4692-aa75-24cd02a8cd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:29.090051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf3f9b-f743-4692-aa75-24cd02a8cd1c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:29.090330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf3f9b-f743-4692-aa75-24cd02a8cd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:29.090600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf3f9b-f743-4692-aa75-24cd02a8cd1c None admin] PATH_INFO: `/v3/domains/b9218cb69cba481bb65884a54c06dc0c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:29.102573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf3f9b-f743-4692-aa75-24cd02a8cd1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=b9218cb69cba481bb65884a54c06dc0c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:29.104048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf3f9b-f743-4692-aa75-24cd02a8cd1c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:29.105667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 575/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Jun 23 08:09:28 2022] GET /identity/v3/domains/b9218cb69cba481bb65884a54c06dc0c => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:29.114550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5ebb3-b44a-4bd5-92f2-3cf96f4b328c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:29.228992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5ebb3-b44a-4bd5-92f2-3cf96f4b328c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:29.264635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350ac7d-4c9d-409e-ab89-ebb86b07f30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:29.265272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ac7d-4c9d-409e-ab89-ebb86b07f30a None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:29.265551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ac7d-4c9d-409e-ab89-ebb86b07f30a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:29.265823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ac7d-4c9d-409e-ab89-ebb86b07f30a None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:29.266541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ac7d-4c9d-409e-ab89-ebb86b07f30a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:29.268044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ac7d-4c9d-409e-ab89-ebb86b07f30a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:29.301310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 598/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:09:29 2022] POST /identity/v3/projects => generated 361 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:09:31.318194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb809267-85b4-4d19-86b6-63bb692e6f94 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:31.319059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb809267-85b4-4d19-86b6-63bb692e6f94 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:31.319274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb809267-85b4-4d19-86b6-63bb692e6f94 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:31.321371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 576/1174] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:31.327685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2e235-bdc2-49bf-bb0e-9e1390666ae4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:31.328407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2e235-bdc2-49bf-bb0e-9e1390666ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:31.328576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2e235-bdc2-49bf-bb0e-9e1390666ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:31.337334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:31.353914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4db2e235-bdc2-49bf-bb0e-9e1390666ae4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:31.355100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4db2e235-bdc2-49bf-bb0e-9e1390666ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:31.426654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 599/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:31.452484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb809267-85b4-4d19-86b6-63bb692e6f94 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:31.575105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb809267-85b4-4d19-86b6-63bb692e6f94 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:31.607695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48f580-0f9c-448a-99bc-a04565a77cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:31.608553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f580-0f9c-448a-99bc-a04565a77cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:31.608828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f580-0f9c-448a-99bc-a04565a77cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:31.609098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f580-0f9c-448a-99bc-a04565a77cb6 None admin] PATH_INFO: `/v3/domains/b9218cb69cba481bb65884a54c06dc0c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:31.610431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48f580-0f9c-448a-99bc-a04565a77cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b9218cb69cba481bb65884a54c06dc0c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:31.611729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48f580-0f9c-448a-99bc-a04565a77cb6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:31.613314 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 577/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Jun 23 08:09:31 2022] GET /identity/v3/domains/b9218cb69cba481bb65884a54c06dc0c => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:31.623023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2e235-bdc2-49bf-bb0e-9e1390666ae4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:31.732697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2e235-bdc2-49bf-bb0e-9e1390666ae4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:31.764861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:31.770920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e912d8-d097-4dbf-a76d-3dfe8870d314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:31.771627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e912d8-d097-4dbf-a76d-3dfe8870d314 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:31.771908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e912d8-d097-4dbf-a76d-3dfe8870d314 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:31.772176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e912d8-d097-4dbf-a76d-3dfe8870d314 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:31.772882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e912d8-d097-4dbf-a76d-3dfe8870d314 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:31.774332 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e912d8-d097-4dbf-a76d-3dfe8870d314 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:31.795651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 600/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:09:31 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:09:33.840779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c362-c3e0-44a9-b991-fcaddf5dff7e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:33.841328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c362-c3e0-44a9-b991-fcaddf5dff7e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:33.841328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c362-c3e0-44a9-b991-fcaddf5dff7e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:33.842605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 578/1178] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:33.848457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a615a8c-9e08-4488-b211-bbc89992b19a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:33.848900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a615a8c-9e08-4488-b211-bbc89992b19a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:33.848992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a615a8c-9e08-4488-b211-bbc89992b19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:33.868409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0a615a8c-9e08-4488-b211-bbc89992b19a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:33.869582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0a615a8c-9e08-4488-b211-bbc89992b19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:33.940524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 601/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:33.964327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823c362-c3e0-44a9-b991-fcaddf5dff7e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:34.017397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:34.088707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823c362-c3e0-44a9-b991-fcaddf5dff7e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:34.128096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cd204-9d24-45a9-9871-2dc936a29fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:34.128885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd204-9d24-45a9-9871-2dc936a29fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:34.129173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd204-9d24-45a9-9871-2dc936a29fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:34.129430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd204-9d24-45a9-9871-2dc936a29fa1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:34.140205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cd204-9d24-45a9-9871-2dc936a29fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c65cd204-9d24-45a9-9871-2dc936a29fa1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:09:34.148190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:09:34.149243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:09:34.150107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 579/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:09:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:09:34.157674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a615a8c-9e08-4488-b211-bbc89992b19a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:34.266323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a615a8c-9e08-4488-b211-bbc89992b19a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:34.288707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:34.311828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:34.312512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:34.312794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:34.313071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:34.314075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:34.314386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:34.315827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:34.324025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 602/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:34.330228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cd204-9d24-45a9-9871-2dc936a29fa1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:34.438075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cd204-9d24-45a9-9871-2dc936a29fa1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:34.444977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:34.473414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130566dc-b370-4a4a-af5e-3f210b572187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:34.474284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130566dc-b370-4a4a-af5e-3f210b572187 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:34.474571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130566dc-b370-4a4a-af5e-3f210b572187 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:34.474846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130566dc-b370-4a4a-af5e-3f210b572187 None admin] PATH_INFO: `/v3/users/b2bfde40febe4c84a2cda428134f290e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:34.489213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130566dc-b370-4a4a-af5e-3f210b572187 None admin] RBAC: Authorizing `identity:get_user(user_id=b2bfde40febe4c84a2cda428134f290e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:34.491572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130566dc-b370-4a4a-af5e-3f210b572187 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:34.494110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 580/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:34 2022] GET /identity/v3/users/b2bfde40febe4c84a2cda428134f290e => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:34.503099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:34.607617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15bcde-cf25-44a1-8682-2ab67cf04aea None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:34.645399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a709-9f9b-4ca0-8991-5e52da2be005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:34.646393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a709-9f9b-4ca0-8991-5e52da2be005 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:34.646832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a709-9f9b-4ca0-8991-5e52da2be005 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:34.647280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a709-9f9b-4ca0-8991-5e52da2be005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:34.706099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:34.726908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1a709-9f9b-4ca0-8991-5e52da2be005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:34.728411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1a709-9f9b-4ca0-8991-5e52da2be005 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:34.830233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 603/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:09:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:09:34.836492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130566dc-b370-4a4a-af5e-3f210b572187 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:34.948375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130566dc-b370-4a4a-af5e-3f210b572187 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:34.992392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5be29f-3b07-4a47-8fbb-f8513c6d6337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:34.993171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5be29f-3b07-4a47-8fbb-f8513c6d6337 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:34.993450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5be29f-3b07-4a47-8fbb-f8513c6d6337 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:34.993724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5be29f-3b07-4a47-8fbb-f8513c6d6337 None admin] PATH_INFO: `/v3/projects/57e7f15b1660464286f982d40a4a1ad1` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:34.995032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5be29f-3b07-4a47-8fbb-f8513c6d6337 None admin] RBAC: Authorizing `identity:get_project(project_id=57e7f15b1660464286f982d40a4a1ad1)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:34.996839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5be29f-3b07-4a47-8fbb-f8513c6d6337 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:34.998559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 581/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:34 2022] GET /identity/v3/projects/57e7f15b1660464286f982d40a4a1ad1 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:35.004924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a709-9f9b-4ca0-8991-5e52da2be005 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:35.088565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:35.113767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a709-9f9b-4ca0-8991-5e52da2be005 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:35.151844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917595a-7f71-4252-b50d-5ba89c51c50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:35.152494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917595a-7f71-4252-b50d-5ba89c51c50a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:35.152777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917595a-7f71-4252-b50d-5ba89c51c50a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:35.153051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917595a-7f71-4252-b50d-5ba89c51c50a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:35.153888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917595a-7f71-4252-b50d-5ba89c51c50a None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=57e7f15b1660464286f982d40a4a1ad1, user.id=b2bfde40febe4c84a2cda428134f290e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:35.154203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917595a-7f71-4252-b50d-5ba89c51c50a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:35.155691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917595a-7f71-4252-b50d-5ba89c51c50a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:35.161824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 604/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:09:35 2022] GET /identity/v3/role_assignments?user.id=b2bfde40febe4c84a2cda428134f290e&scope.project.id=57e7f15b1660464286f982d40a4a1ad1&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:37.153102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84097c-8b90-4778-91d3-8dd37ac30278 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:37.153972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84097c-8b90-4778-91d3-8dd37ac30278 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:37.154143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84097c-8b90-4778-91d3-8dd37ac30278 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:37.156368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 582/1186] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:37.162636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7a17-bdf2-473b-9366-b5cca15874e4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:37.163095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7a17-bdf2-473b-9366-b5cca15874e4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:37.163188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7a17-bdf2-473b-9366-b5cca15874e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:37.185098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-746d7a17-bdf2-473b-9366-b5cca15874e4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:37.186281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-746d7a17-bdf2-473b-9366-b5cca15874e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:37.263068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 605/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:37.299743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84097c-8b90-4778-91d3-8dd37ac30278 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:37.338157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:37.417741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84097c-8b90-4778-91d3-8dd37ac30278 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:37.458495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0203a-ca14-4589-83eb-ff79737446c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:37.459691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0203a-ca14-4589-83eb-ff79737446c6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:37.460050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0203a-ca14-4589-83eb-ff79737446c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:37.460433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0203a-ca14-4589-83eb-ff79737446c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:37.475244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0203a-ca14-4589-83eb-ff79737446c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-43a0203a-ca14-4589-83eb-ff79737446c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:09:37.482833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:09:37.483976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:09:37.484832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 583/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:09:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:09:37.493475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d7a17-bdf2-473b-9366-b5cca15874e4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:37.574461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:37.609948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d7a17-bdf2-473b-9366-b5cca15874e4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:37.646724 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:37.647480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:37.647761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:37.648030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:37.648771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:37.649083 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:37.650472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:37.658842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 606/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:09:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:37.664981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0203a-ca14-4589-83eb-ff79737446c6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:37.758639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:37.769577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0203a-ca14-4589-83eb-ff79737446c6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:37.810991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f491f-bae6-4a0e-b270-13f231b9d870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:37.812310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f491f-bae6-4a0e-b270-13f231b9d870 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:37.812661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f491f-bae6-4a0e-b270-13f231b9d870 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:37.813022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f491f-bae6-4a0e-b270-13f231b9d870 None admin] PATH_INFO: `/v3/users/b2bfde40febe4c84a2cda428134f290e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:37.815035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f491f-bae6-4a0e-b270-13f231b9d870 None admin] RBAC: Authorizing `identity:get_user(user_id=b2bfde40febe4c84a2cda428134f290e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:37.817360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f491f-bae6-4a0e-b270-13f231b9d870 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:37.819927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 584/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:37 2022] GET /identity/v3/users/b2bfde40febe4c84a2cda428134f290e => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:37.828456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:37.949326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49acee-57cf-42e9-9139-ab52516d3b2a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:37.984673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511faa-f758-499a-b4a8-e25028a132a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:37.985630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511faa-f758-499a-b4a8-e25028a132a4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:37.985872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511faa-f758-499a-b4a8-e25028a132a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:37.986114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511faa-f758-499a-b4a8-e25028a132a4 None admin] PATH_INFO: `/v3/projects/57e7f15b1660464286f982d40a4a1ad1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:37.987425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b511faa-f758-499a-b4a8-e25028a132a4 None admin] RBAC: Authorizing `identity:get_project(project_id=57e7f15b1660464286f982d40a4a1ad1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:37.988870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b511faa-f758-499a-b4a8-e25028a132a4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:37.990547 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 607/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:37 2022] GET /identity/v3/projects/57e7f15b1660464286f982d40a4a1ad1 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:09:38.000304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f491f-bae6-4a0e-b270-13f231b9d870 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:38.109994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f491f-bae6-4a0e-b270-13f231b9d870 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:38.151927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42a2e-76ea-4645-a52a-d5c6bc9b3d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:38.152813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42a2e-76ea-4645-a52a-d5c6bc9b3d96 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:38.153093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42a2e-76ea-4645-a52a-d5c6bc9b3d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:38.153365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42a2e-76ea-4645-a52a-d5c6bc9b3d96 None admin] PATH_INFO: `/v3/projects/57e7f15b1660464286f982d40a4a1ad1/users/b2bfde40febe4c84a2cda428134f290e/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:38.155345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42a2e-76ea-4645-a52a-d5c6bc9b3d96 None admin] RBAC: Authorizing `identity:create_grant(project_id=57e7f15b1660464286f982d40a4a1ad1, user_id=b2bfde40febe4c84a2cda428134f290e, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:38.156776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42a2e-76ea-4645-a52a-d5c6bc9b3d96 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:38.178475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 585/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:09:37 2022] PUT /identity/v3/projects/57e7f15b1660464286f982d40a4a1ad1/users/b2bfde40febe4c84a2cda428134f290e/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:09:40.130417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1169e4-20e0-44c4-90c7-c4e69a1027c0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:40.131016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1169e4-20e0-44c4-90c7-c4e69a1027c0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:40.131016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1169e4-20e0-44c4-90c7-c4e69a1027c0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:40.132393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 608/1193] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:09:40.138614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d015-4533-4847-b507-2ea96c6fc602 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:40.139400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d015-4533-4847-b507-2ea96c6fc602 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:40.139523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d015-4533-4847-b507-2ea96c6fc602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:40.158264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9d84d015-4533-4847-b507-2ea96c6fc602 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:09:40.159487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9d84d015-4533-4847-b507-2ea96c6fc602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:09:40.173907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:40.229396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 586/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:09:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:09:40.253842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1169e4-20e0-44c4-90c7-c4e69a1027c0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:40.259868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:40.368966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1169e4-20e0-44c4-90c7-c4e69a1027c0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:40.403956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da7359-c8f5-44c8-8a29-0d827cfd5bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:40.404674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7359-c8f5-44c8-8a29-0d827cfd5bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:40.404955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7359-c8f5-44c8-8a29-0d827cfd5bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:40.405229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7359-c8f5-44c8-8a29-0d827cfd5bd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:40.416893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da7359-c8f5-44c8-8a29-0d827cfd5bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c8da7359-c8f5-44c8-8a29-0d827cfd5bd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:09:40.425170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:09:40.426143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:09:40.427314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 609/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:09:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:09:40.436285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84d015-4533-4847-b507-2ea96c6fc602 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:40.544878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84d015-4533-4847-b507-2ea96c6fc602 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:40.580351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:40.581011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:40.581294 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:40.581569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:40.582318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:40.582624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:40.584030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:40.592327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 587/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:09:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:40.599147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da7359-c8f5-44c8-8a29-0d827cfd5bd2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:40.670145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:40.708512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da7359-c8f5-44c8-8a29-0d827cfd5bd2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:40.743829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a88fa2-74d1-4476-9bd6-89055e55e13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:40.744685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a88fa2-74d1-4476-9bd6-89055e55e13c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:40.745309 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a88fa2-74d1-4476-9bd6-89055e55e13c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:40.745582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a88fa2-74d1-4476-9bd6-89055e55e13c None admin] PATH_INFO: `/v3/users/b2bfde40febe4c84a2cda428134f290e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:40.746805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a88fa2-74d1-4476-9bd6-89055e55e13c None admin] RBAC: Authorizing `identity:get_user(user_id=b2bfde40febe4c84a2cda428134f290e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:40.748284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a88fa2-74d1-4476-9bd6-89055e55e13c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:40.749915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 610/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 08:09:40 2022] GET /identity/v3/users/b2bfde40febe4c84a2cda428134f290e => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:40.756039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:40.773339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:40.876568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a88e8-0265-494e-9619-a7836fb4a0f6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:40.911631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc034b3c-437f-4987-a309-58ecb6e44527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:40.912275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc034b3c-437f-4987-a309-58ecb6e44527 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:40.912566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc034b3c-437f-4987-a309-58ecb6e44527 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:40.912834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc034b3c-437f-4987-a309-58ecb6e44527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:40.990891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc034b3c-437f-4987-a309-58ecb6e44527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:40.992301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc034b3c-437f-4987-a309-58ecb6e44527 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:41.007998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:41.099414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 588/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:09:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:09:41.105573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a88fa2-74d1-4476-9bd6-89055e55e13c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:41.213622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a88fa2-74d1-4476-9bd6-89055e55e13c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:41.243201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:41.250082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b42e71-9ef9-48ad-ac97-779ab8b0c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:41.251554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b42e71-9ef9-48ad-ac97-779ab8b0c35a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:41.251841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b42e71-9ef9-48ad-ac97-779ab8b0c35a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:41.252102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b42e71-9ef9-48ad-ac97-779ab8b0c35a None admin] PATH_INFO: `/v3/projects/57e7f15b1660464286f982d40a4a1ad1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:41.253392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b42e71-9ef9-48ad-ac97-779ab8b0c35a None admin] RBAC: Authorizing `identity:get_project(project_id=57e7f15b1660464286f982d40a4a1ad1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:41.254875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b42e71-9ef9-48ad-ac97-779ab8b0c35a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:41.256674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 611/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 08:09:41 2022] GET /identity/v3/projects/57e7f15b1660464286f982d40a4a1ad1 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:41.263569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc034b3c-437f-4987-a309-58ecb6e44527 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:09:41.373595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc034b3c-437f-4987-a309-58ecb6e44527 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:09:41.387714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:09:41.412461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774453c-47ca-4aee-b587-5646b8323212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:09:41.413104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774453c-47ca-4aee-b587-5646b8323212 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:09:41.413390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774453c-47ca-4aee-b587-5646b8323212 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:09:41.413663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774453c-47ca-4aee-b587-5646b8323212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:09:41.414485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774453c-47ca-4aee-b587-5646b8323212 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=57e7f15b1660464286f982d40a4a1ad1, user.id=b2bfde40febe4c84a2cda428134f290e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:09:41.414794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774453c-47ca-4aee-b587-5646b8323212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:09:41.416248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774453c-47ca-4aee-b587-5646b8323212 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:09:41.423281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 589/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:09:41 2022] GET /identity/v3/role_assignments?user.id=b2bfde40febe4c84a2cda428134f290e&scope.project.id=57e7f15b1660464286f982d40a4a1ad1&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:09:51.419745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=12 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:36.303797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e469ec-e514-4312-ae8d-0557b38b7cb7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:36.304421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e469ec-e514-4312-ae8d-0557b38b7cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:36.304421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e469ec-e514-4312-ae8d-0557b38b7cb7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:36.305725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 612/1201] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:10:36.311401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde419-8bde-436d-bb9a-127b9aa8734a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:36.311875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde419-8bde-436d-bb9a-127b9aa8734a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:36.312108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde419-8bde-436d-bb9a-127b9aa8734a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:36.332898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8ffde419-8bde-436d-bb9a-127b9aa8734a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:10:36.334414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8ffde419-8bde-436d-bb9a-127b9aa8734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:10:36.419256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 590/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:10:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:10:36.445372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e469ec-e514-4312-ae8d-0557b38b7cb7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:36.579648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e469ec-e514-4312-ae8d-0557b38b7cb7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:36.616419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:36.617205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:36.617482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:36.617752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:36.631079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:36.632580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:36.651075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:36.652286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:36.653676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:36.653676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:36.653676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:36.653676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:36.653676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 613/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:36.660410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde419-8bde-436d-bb9a-127b9aa8734a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:36.799022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde419-8bde-436d-bb9a-127b9aa8734a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:36.835849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:36.845069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426cdc4-c2ee-4646-bf0f-f12fe1479c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:36.846115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426cdc4-c2ee-4646-bf0f-f12fe1479c98 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:36.846459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426cdc4-c2ee-4646-bf0f-f12fe1479c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:36.846824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426cdc4-c2ee-4646-bf0f-f12fe1479c98 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:36.847875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426cdc4-c2ee-4646-bf0f-f12fe1479c98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:36.848309 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426cdc4-c2ee-4646-bf0f-f12fe1479c98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:36.850119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426cdc4-c2ee-4646-bf0f-f12fe1479c98 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:36.863815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 591/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:36.870109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:36.936981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:37.003825 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3456fd-b79e-49ef-98c3-39584e55a737 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:37.051355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e175d7-51d2-44ea-88ca-c9f9ef66974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:37.052552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e175d7-51d2-44ea-88ca-c9f9ef66974a None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:37.052877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e175d7-51d2-44ea-88ca-c9f9ef66974a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:37.053162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e175d7-51d2-44ea-88ca-c9f9ef66974a None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:37.053886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e175d7-51d2-44ea-88ca-c9f9ef66974a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:37.058163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e175d7-51d2-44ea-88ca-c9f9ef66974a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:37.092660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 614/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 08:10:36 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:10:39.272260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dee1d-9e79-49ab-b7f2-4cf252fa2751 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:39.272862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dee1d-9e79-49ab-b7f2-4cf252fa2751 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:39.272862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dee1d-9e79-49ab-b7f2-4cf252fa2751 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:39.274268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 592/1206] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:10:39.280539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1925727-dd6c-4fe7-abb7-16aa14c9c4c7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:39.281047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1925727-dd6c-4fe7-abb7-16aa14c9c4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:39.281130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1925727-dd6c-4fe7-abb7-16aa14c9c4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:39.299777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c1925727-dd6c-4fe7-abb7-16aa14c9c4c7 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:10:39.301060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c1925727-dd6c-4fe7-abb7-16aa14c9c4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:10:39.380478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 615/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:10:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:10:39.406070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dee1d-9e79-49ab-b7f2-4cf252fa2751 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:39.534060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dee1d-9e79-49ab-b7f2-4cf252fa2751 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:39.575390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16633c-5398-49f8-812d-428af0310820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:39.576708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16633c-5398-49f8-812d-428af0310820 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:39.577068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16633c-5398-49f8-812d-428af0310820 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:39.577346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16633c-5398-49f8-812d-428af0310820 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:39.590835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16633c-5398-49f8-812d-428af0310820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-af16633c-5398-49f8-812d-428af0310820 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:10:39.598975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:39.600983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:39.602114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 593/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:39.606430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1925727-dd6c-4fe7-abb7-16aa14c9c4c7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:39.658630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:39.727106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1925727-dd6c-4fe7-abb7-16aa14c9c4c7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:39.766158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:39.766861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:39.767138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:39.767443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:39.768190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:39.768497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:39.771323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:39.780593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 616/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:10:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:39.788300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16633c-5398-49f8-812d-428af0310820 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:39.847414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:39.907538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16633c-5398-49f8-812d-428af0310820 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:39.944563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:39.945461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:39.945767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:39.946067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:39.958655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:39.960330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:39.978259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:39.980337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:39.981292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:39.981292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:39.981292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:39.981292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:39.981292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 594/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:10:39.985448 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:40.101751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:40.103036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23068-159d-40bb-8c5e-ae37079e5a6d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:40.140436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:40.141132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:40.141426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:40.141681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:40.142402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:40.142703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:40.144120 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:40.157368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 617/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:40.163180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:40.274449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5fafb-a89c-4fc6-975b-e581e9fa5dd3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:40.276057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:40.312626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:40.313416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:40.313698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:40.313968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:40.321595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:40.323089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:10:40.340106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:40.341624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:40.342637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 595/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:40.347935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:40.461237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2845e-d20b-41d4-a224-4b1f9d13db1b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:40.501262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:40.501945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:40.502273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:40.502495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:40.503379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:40.508204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:40.508204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:40.526971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 618/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 08:10:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:40.532869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:40.645145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa0c28-41a4-4020-b52c-e802ad9477f7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:40.691295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af51aa-ced5-4261-b187-31249776c691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:40.692004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af51aa-ced5-4261-b187-31249776c691 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:40.692282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af51aa-ced5-4261-b187-31249776c691 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:40.693269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af51aa-ced5-4261-b187-31249776c691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:40.729130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:40.776795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af51aa-ced5-4261-b187-31249776c691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:40.778424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af51aa-ced5-4261-b187-31249776c691 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:40.895993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 596/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:10:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:10:40.901792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:40.925867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:41.026081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cf0c9-0d94-40c7-a37e-8bea88ca37ad None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:41.064856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:41.065752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:41.065955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:41.066206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:41.077405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:41.078855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:41.096936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:41.097918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:41.099398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:41.099398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:41.099398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:41.099398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:41.099398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 619/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:41.103983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af51aa-ced5-4261-b187-31249776c691 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:41.169448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:41.211743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af51aa-ced5-4261-b187-31249776c691 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:41.254792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:41.255507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:41.255790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:41.256061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:41.257013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:41.257154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:41.258659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:41.277248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 597/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:41.278322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:41.356222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:41.395527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b620c-84f9-4ecb-bd1d-8f2affe40e6a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:41.433840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:41.434700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:41.435010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:41.435555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:41.443358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:41.444970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:10:41.459910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:41.460936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:41.462408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 620/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:10:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:10:41.467396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:41.582578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52785d9-d49d-4cb4-a410-ca0ae732bcd9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:41.610887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:41.624874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e848d8c-01a5-4de6-8391-5cf928e7511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:41.625633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e848d8c-01a5-4de6-8391-5cf928e7511e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:41.625985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e848d8c-01a5-4de6-8391-5cf928e7511e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:41.626322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e848d8c-01a5-4de6-8391-5cf928e7511e None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:41.627192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e848d8c-01a5-4de6-8391-5cf928e7511e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:41.627593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e848d8c-01a5-4de6-8391-5cf928e7511e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:41.629044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e848d8c-01a5-4de6-8391-5cf928e7511e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:41.649780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 598/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:10:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:41.651381 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:41.774155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ab2b4-49d5-410c-8ba1-26c60c885adf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:41.814848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:41.821666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95996f-1a36-48fd-b032-256abe8f759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:41.822330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95996f-1a36-48fd-b032-256abe8f759b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:41.822608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95996f-1a36-48fd-b032-256abe8f759b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:41.822878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95996f-1a36-48fd-b032-256abe8f759b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:41.823764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95996f-1a36-48fd-b032-256abe8f759b None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=b93c07dd32f543a69e74599297cbe17a) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:41.824543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95996f-1a36-48fd-b032-256abe8f759b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:41.825646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95996f-1a36-48fd-b032-256abe8f759b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:41.844414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 621/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:10:41 2022] GET /identity/v3/role_assignments?user.id=b93c07dd32f543a69e74599297cbe17a&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:43.922909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251caf2-14c5-4cb3-92d4-2aa05add9c9d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:43.923556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251caf2-14c5-4cb3-92d4-2aa05add9c9d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:43.923556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251caf2-14c5-4cb3-92d4-2aa05add9c9d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:43.924883 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 599/1220] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:10:43.930482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc822a5-6f93-4b9b-a71e-751117a12abd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:43.930944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc822a5-6f93-4b9b-a71e-751117a12abd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:43.931046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc822a5-6f93-4b9b-a71e-751117a12abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:43.948067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0bc822a5-6f93-4b9b-a71e-751117a12abd None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:10:43.949242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0bc822a5-6f93-4b9b-a71e-751117a12abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:10:44.028221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 622/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:10:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:10:44.051196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251caf2-14c5-4cb3-92d4-2aa05add9c9d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:44.157869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251caf2-14c5-4cb3-92d4-2aa05add9c9d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:44.192207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858132bc-f03b-4bb6-9cd8-b7501292a219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:44.192923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858132bc-f03b-4bb6-9cd8-b7501292a219 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:44.193202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858132bc-f03b-4bb6-9cd8-b7501292a219 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:44.193549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858132bc-f03b-4bb6-9cd8-b7501292a219 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:44.205576 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858132bc-f03b-4bb6-9cd8-b7501292a219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-858132bc-f03b-4bb6-9cd8-b7501292a219 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:10:44.213973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:44.216587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:44.219063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 600/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:10:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:44.222600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc822a5-6f93-4b9b-a71e-751117a12abd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:44.337646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc822a5-6f93-4b9b-a71e-751117a12abd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:44.360772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:44.374190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:44.374935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:44.375714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:44.376005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:44.376773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:44.377076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:44.378474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:44.388455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 623/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:10:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:44.394025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858132bc-f03b-4bb6-9cd8-b7501292a219 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:44.442617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:44.509267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858132bc-f03b-4bb6-9cd8-b7501292a219 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:44.546756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:44.547613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:44.547903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:44.548177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:44.560107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:44.561908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:44.588767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:44.589863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:44.591226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:44.591226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:44.591226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:44.591226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:44.591226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 601/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:44.597822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:44.709885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097aeb-4471-431c-9e45-6106ee72e20b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:44.748225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:44.748927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:44.749163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:44.749456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:44.750202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:44.750445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:44.751859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:44.764945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 624/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:44.770920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:44.883234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:44.890664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e571b-0c38-44c9-a1df-1cfa2e4bf3e0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:44.926831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163750e-8329-431a-addf-bd225af37920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:44.927803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163750e-8329-431a-addf-bd225af37920 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:44.928092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163750e-8329-431a-addf-bd225af37920 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:44.928465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163750e-8329-431a-addf-bd225af37920 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:44.936762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163750e-8329-431a-addf-bd225af37920 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:44.938265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163750e-8329-431a-addf-bd225af37920 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0163750e-8329-431a-addf-bd225af37920 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:10:44.952452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:44.953449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:44.954421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 602/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:44.959817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:45.011250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:45.067082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cfd93-fac4-4276-b4e5-692702390814 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:45.101523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:45.102193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:45.102467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:45.102733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:45.103604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:45.103897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:45.105369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:45.122917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 625/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 08:10:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:45.129103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163750e-8329-431a-addf-bd225af37920 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:45.238924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163750e-8329-431a-addf-bd225af37920 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:45.273157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:45.273945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:45.274218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:45.274487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:45.285057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:45.286497 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:45.304012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:45.305139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:45.306256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:45.306256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:45.306256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:45.306256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:45.306256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 603/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:45.311400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:45.417077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56c5f0-2492-4470-9665-65422548a71a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:45.419460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:45.458484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:45.459157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:45.459477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:45.459756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:45.460582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:45.461006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:45.462804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:45.475955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 626/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:10:45.481822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:45.492412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:45.595240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf2824-95fa-456c-93be-5305551a5ab4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:45.632767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:45.633565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:45.633849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:45.634124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:45.641346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:45.642805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:10:45.657222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:45.659481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:45.661154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 604/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:10:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:45.667172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:45.780733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e46538-62ed-40b1-afaf-97bd33d29e06 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:45.824162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a2ba3-a6c6-4cac-9d87-475c84adc15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:45.824846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a2ba3-a6c6-4cac-9d87-475c84adc15e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:45.825253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a2ba3-a6c6-4cac-9d87-475c84adc15e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:45.825767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a2ba3-a6c6-4cac-9d87-475c84adc15e None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:45.827284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a2ba3-a6c6-4cac-9d87-475c84adc15e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:45.827829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a2ba3-a6c6-4cac-9d87-475c84adc15e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:45.830363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a2ba3-a6c6-4cac-9d87-475c84adc15e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:45.845116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 627/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:10:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:45.856742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:45.927482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:45.967014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dbaa2-af1c-45a3-9c64-29a5597f4b1d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:46.006619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5c1d-ff3a-4aea-9b09-5adc3bd033ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:46.007553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c1d-ff3a-4aea-9b09-5adc3bd033ae None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:46.007828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c1d-ff3a-4aea-9b09-5adc3bd033ae None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:46.008106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c1d-ff3a-4aea-9b09-5adc3bd033ae None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/b93c07dd32f543a69e74599297cbe17a/roles/fa8c35ad77b442fbbc0d20d458fe8731` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:46.024650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a5c1d-ff3a-4aea-9b09-5adc3bd033ae None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=b93c07dd32f543a69e74599297cbe17a, role_id=fa8c35ad77b442fbbc0d20d458fe8731)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:46.027546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a5c1d-ff3a-4aea-9b09-5adc3bd033ae None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:46.042720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 605/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:10:45 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/b93c07dd32f543a69e74599297cbe17a/roles/fa8c35ad77b442fbbc0d20d458fe8731 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:10:48.218739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722987-b479-4118-a724-932469bf203b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:48.219361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722987-b479-4118-a724-932469bf203b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:48.219361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722987-b479-4118-a724-932469bf203b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:48.220695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 628/1233] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:10:48.226112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8318c-47dd-4bcc-8e9f-0f46bba393e9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:48.226548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8318c-47dd-4bcc-8e9f-0f46bba393e9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:48.226695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8318c-47dd-4bcc-8e9f-0f46bba393e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:48.245105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b8d8318c-47dd-4bcc-8e9f-0f46bba393e9 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:10:48.246328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b8d8318c-47dd-4bcc-8e9f-0f46bba393e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:10:48.315335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 606/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:10:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 08:10:48.339071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d722987-b479-4118-a724-932469bf203b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:48.369159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:48.453463 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d722987-b479-4118-a724-932469bf203b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:48.492743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2ce40-ea6b-4539-9e5f-dd849a80bc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:48.493881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2ce40-ea6b-4539-9e5f-dd849a80bc3d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:48.494179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2ce40-ea6b-4539-9e5f-dd849a80bc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:48.494490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2ce40-ea6b-4539-9e5f-dd849a80bc3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:48.506269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2ce40-ea6b-4539-9e5f-dd849a80bc3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4bb2ce40-ea6b-4539-9e5f-dd849a80bc3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:10:48.514219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:48.515416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:48.516560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 629/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 08:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:48.521802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8318c-47dd-4bcc-8e9f-0f46bba393e9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:48.573191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:48.629156 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8318c-47dd-4bcc-8e9f-0f46bba393e9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:48.664863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:48.665568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:48.665964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:48.666247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:48.667157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:48.667509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:48.669056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:48.678152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 607/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 08:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:10:48.684065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2ce40-ea6b-4539-9e5f-dd849a80bc3d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:48.776510 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:48.794798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2ce40-ea6b-4539-9e5f-dd849a80bc3d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:48.833153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:48.833956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:48.834236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:48.834506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:48.846401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:48.847917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:48.866033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:48.867157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:48.868116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:48.868116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:48.868116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:48.868116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:48.868116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 630/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:10:48.873635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:48.978600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a437a-bbdf-4d0d-aa81-e31f096e0237 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:48.988260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:49.015573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:49.016243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:49.016524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:49.016793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:49.017512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:49.017817 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:49.019238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:49.032150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 608/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:49.037934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:49.144817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee6c36-5287-4237-8182-75963094c87d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:49.179645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:49.180591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:49.180911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:49.181221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:49.184350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:49.190756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:49.192466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:10:49.209066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:49.210089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:49.211105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 631/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:49.216916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:49.335263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc2de1-06ac-4043-8448-3a80d2223a27 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:49.373213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:49.373919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:49.374202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:49.374479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:49.375363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:49.375701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:49.377160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:49.396393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 609/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 08:10:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:49.402635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:49.513347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f682-9c1d-49d7-b3ee-418c4c4b4bfd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:49.549017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0400f4-f3d7-40e5-8858-8168ac5f117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:49.549742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0400f4-f3d7-40e5-8858-8168ac5f117a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:49.549960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0400f4-f3d7-40e5-8858-8168ac5f117a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:49.550234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0400f4-f3d7-40e5-8858-8168ac5f117a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:49.619535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:49.627989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0400f4-f3d7-40e5-8858-8168ac5f117a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:49.629764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0400f4-f3d7-40e5-8858-8168ac5f117a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:49.740291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 632/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:10:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:10:49.745816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:49.776476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:49.861004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba5733-039f-4b01-85a3-e398c7789186 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:49.897913 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:49.898637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:49.898916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:49.899191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:49.909713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:49.911291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:49.928986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:49.930214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:49.931327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:49.931327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:49.931327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:49.931327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:49.931327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 610/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:49.936156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0400f4-f3d7-40e5-8858-8168ac5f117a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:50.042357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:50.049131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0400f4-f3d7-40e5-8858-8168ac5f117a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:50.088354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:50.089120 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:50.089398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:50.089700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:50.090520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:50.091343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:50.093524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:50.109060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 633/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:50.115696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:50.200504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:50.226961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bb08d-3b1c-4adb-9228-8449d824e1fa None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:50.266067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:50.266954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:50.267391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:50.267664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:50.274850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:50.276374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:10:50.290372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:50.291687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:50.292950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 611/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:50.297602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:50.408135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318cd05-ddf5-46ad-804d-9d9e7540cf0c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:50.446839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84c4ec-f8ec-4830-a521-50c606bec6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:50.447987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84c4ec-f8ec-4830-a521-50c606bec6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:50.448271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84c4ec-f8ec-4830-a521-50c606bec6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:50.448550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84c4ec-f8ec-4830-a521-50c606bec6d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:50.449400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84c4ec-f8ec-4830-a521-50c606bec6d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:50.449719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84c4ec-f8ec-4830-a521-50c606bec6d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:50.451453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84c4ec-f8ec-4830-a521-50c606bec6d7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:50.465460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 634/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:50.473881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:50.585035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845cdb5-60cd-4a66-b4e9-b1a0d4efd6f7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:50.621583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965ec98-481a-4fb0-9463-0ccc726a9c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:50.622265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965ec98-481a-4fb0-9463-0ccc726a9c73 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:50.622606 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965ec98-481a-4fb0-9463-0ccc726a9c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:50.622814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965ec98-481a-4fb0-9463-0ccc726a9c73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:50.623660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965ec98-481a-4fb0-9463-0ccc726a9c73 None admin] RBAC: Adding query filter params (role.id=fa8c35ad77b442fbbc0d20d458fe8731, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=b93c07dd32f543a69e74599297cbe17a) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:50.623971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965ec98-481a-4fb0-9463-0ccc726a9c73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:50.625399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965ec98-481a-4fb0-9463-0ccc726a9c73 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:50.631285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 612/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:10:50 2022] GET /identity/v3/role_assignments?user.id=b93c07dd32f543a69e74599297cbe17a&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=fa8c35ad77b442fbbc0d20d458fe8731 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:10:52.684744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc1954-ad2d-4b45-af9e-4eece82cdf9e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:52.685348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc1954-ad2d-4b45-af9e-4eece82cdf9e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:52.685348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc1954-ad2d-4b45-af9e-4eece82cdf9e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:52.687015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 635/1247] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:10:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:10:52.692374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c50c3-fdb4-4f07-ab12-7672b748ae2a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:52.692823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c50c3-fdb4-4f07-ab12-7672b748ae2a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:52.692925 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c50c3-fdb4-4f07-ab12-7672b748ae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:52.698094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:52.714544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1b7c50c3-fdb4-4f07-ab12-7672b748ae2a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:10:52.715999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1b7c50c3-fdb4-4f07-ab12-7672b748ae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:10:52.795356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 613/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:10:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:10:52.815020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc1954-ad2d-4b45-af9e-4eece82cdf9e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:52.835237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:52.928875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc1954-ad2d-4b45-af9e-4eece82cdf9e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:52.967243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ea26-b7bd-484e-aa24-7177527324ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:52.968140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ea26-b7bd-484e-aa24-7177527324ea None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:52.968421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ea26-b7bd-484e-aa24-7177527324ea None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:52.968699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ea26-b7bd-484e-aa24-7177527324ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:52.980689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442ea26-b7bd-484e-aa24-7177527324ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7442ea26-b7bd-484e-aa24-7177527324ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:10:52.989113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:52.990206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:52.991336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 636/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:10:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:52.997081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c50c3-fdb4-4f07-ab12-7672b748ae2a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:53.110808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c50c3-fdb4-4f07-ab12-7672b748ae2a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:53.142727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:53.149839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:53.150835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:53.151298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:53.151609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:53.152471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:53.152893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:53.154709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:53.165650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 614/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:10:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:53.171698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ea26-b7bd-484e-aa24-7177527324ea None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:53.269213 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:53.295382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ea26-b7bd-484e-aa24-7177527324ea None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:53.334488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:53.335313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:53.335601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:53.335870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:53.347835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:53.349312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:53.367073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:53.368659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:53.369678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:53.369678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:53.369678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:53.369678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:53.373649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 637/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:53.374550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:53.487864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334856a-60b0-4212-9a75-4bdfabdfbb2d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:53.531536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:53.532201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:53.532485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:53.532760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:53.533479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:53.533785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:53.535197 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:53.547263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 615/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:53.552899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:53.672408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c3a16-0441-4714-871c-b05939423fb7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:53.701727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:53.711352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:53.712238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:53.712534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:53.712856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:53.721855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:53.723460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:10:53.739020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:53.740255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:53.741793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 638/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:53.746725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:53.809751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:53.860267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaadd5-bb42-4421-9fb4-60cd878171ef None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:53.900001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:53.901040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:53.901374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:53.901862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:53.902823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:53.903284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:53.904962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:53.924492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 616/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 08:10:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:53.930344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:54.052339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c43d61-edb4-47e7-9984-0e37a6f9a7d3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:54.088692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304fe5d-16d0-41ec-9711-d467c02a6b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:54.089439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fe5d-16d0-41ec-9711-d467c02a6b15 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:54.089741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fe5d-16d0-41ec-9711-d467c02a6b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:54.090012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fe5d-16d0-41ec-9711-d467c02a6b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:54.157844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:54.179162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304fe5d-16d0-41ec-9711-d467c02a6b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:54.180713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304fe5d-16d0-41ec-9711-d467c02a6b15 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:54.301191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 639/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:10:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:10:54.307069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:54.415276 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833e828-6d20-4ccf-a3d4-ce4d92057279 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:54.430128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:54.454534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:54.455452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:54.455732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:54.456078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:54.468127 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:54.469895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:54.489935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:54.491112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:54.492470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:54.492470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:54.492470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:54.492470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:54.492470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 617/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:54.498169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304fe5d-16d0-41ec-9711-d467c02a6b15 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:54.596000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:54.610693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304fe5d-16d0-41ec-9711-d467c02a6b15 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:54.653874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:54.655503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:54.655870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:54.656181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:54.657039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:54.657348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:54.658918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:54.673204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 640/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:10:54.679247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:54.795934 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932f46e-9073-4658-9bbf-b82137343176 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:54.833057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:54.834315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:54.834659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:54.835110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:54.844714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:54.846402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:10:54.862578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:54.863616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:10:54.864534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 618/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:10:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:54.870752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:54.977853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43b29c-c415-40d5-a787-a11e849edcb6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:55.014401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a200-55b6-4d6e-8ace-4546e7c88c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:55.015052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a200-55b6-4d6e-8ace-4546e7c88c88 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:55.015367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a200-55b6-4d6e-8ace-4546e7c88c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:55.015644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a200-55b6-4d6e-8ace-4546e7c88c88 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:55.016438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a200-55b6-4d6e-8ace-4546e7c88c88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:55.016743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a200-55b6-4d6e-8ace-4546e7c88c88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:55.018206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a200-55b6-4d6e-8ace-4546e7c88c88 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:55.033521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 641/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:55.039793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:55.068040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:55.155271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6177159-996f-405b-8d3a-c03921cae9fc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:55.190785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525c09-29bb-4fd4-a9c1-303520482e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:55.191489 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525c09-29bb-4fd4-a9c1-303520482e47 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:55.191778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525c09-29bb-4fd4-a9c1-303520482e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:55.192058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525c09-29bb-4fd4-a9c1-303520482e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:55.192893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525c09-29bb-4fd4-a9c1-303520482e47 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=b93c07dd32f543a69e74599297cbe17a) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:55.193205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525c09-29bb-4fd4-a9c1-303520482e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:55.194648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525c09-29bb-4fd4-a9c1-303520482e47 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:55.201291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 619/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:10:55 2022] GET /identity/v3/role_assignments?user.id=b93c07dd32f543a69e74599297cbe17a&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:10:57.282037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfc2ec-3300-463e-9478-822d66339d64 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:57.282773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfc2ec-3300-463e-9478-822d66339d64 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:57.282773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfc2ec-3300-463e-9478-822d66339d64 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:57.284024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 642/1261] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:10:57.289154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537278e-7df4-4f67-a0d1-43d42caeca39 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:57.289564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537278e-7df4-4f67-a0d1-43d42caeca39 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:57.289708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537278e-7df4-4f67-a0d1-43d42caeca39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:57.306616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e537278e-7df4-4f67-a0d1-43d42caeca39 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:10:57.307820 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e537278e-7df4-4f67-a0d1-43d42caeca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:10:57.377793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 620/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:10:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:10:57.401475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfc2ec-3300-463e-9478-822d66339d64 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:57.418900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:57.527085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfc2ec-3300-463e-9478-822d66339d64 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:57.566141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4229bc-4d0f-4b21-9bdb-20e6d8261615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:57.566875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4229bc-4d0f-4b21-9bdb-20e6d8261615 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:57.567152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4229bc-4d0f-4b21-9bdb-20e6d8261615 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:57.567460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4229bc-4d0f-4b21-9bdb-20e6d8261615 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:57.580042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4229bc-4d0f-4b21-9bdb-20e6d8261615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ca4229bc-4d0f-4b21-9bdb-20e6d8261615 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:10:57.587669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:57.588754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:57.589739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 643/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:10:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:57.595474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537278e-7df4-4f67-a0d1-43d42caeca39 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:57.613307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:57.697335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537278e-7df4-4f67-a0d1-43d42caeca39 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:57.732333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:57.732986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:57.733268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:57.733540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:57.734450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:57.734756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:57.736154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:57.744874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 621/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:10:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:57.751696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4229bc-4d0f-4b21-9bdb-20e6d8261615 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:57.834131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:57.873360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4229bc-4d0f-4b21-9bdb-20e6d8261615 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:57.912622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:57.913749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:57.914124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:57.914478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:57.927193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:57.928707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:57.946205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:57.947232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:57.952316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:57.952316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:57.952316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:57.952316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:57.952316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 644/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:57.953386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:58.044581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:58.067002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f75c3b-4f01-46e8-9af2-eccc028aa667 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:58.103560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:58.104142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:58.104420 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:58.104695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:58.105416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:58.105725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:58.107155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:58.119419 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 622/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:58.125949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:58.252009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd0d58-37eb-4589-87c6-95b273563cfa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:58.278670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:58.301496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:58.302318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:58.302598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:58.302869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:58.311348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:58.312798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:10:58.327747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:58.328942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:58.329978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 645/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:58.343250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:58.456648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65964d24-7135-4df3-af43-cd51874f85f2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:58.495545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:58.496217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:58.496502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:58.496836 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:58.497747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:58.498062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:58.499534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:58.508714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:58.520575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 623/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 08:10:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:10:58.526366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:58.631870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1918-0861-4364-bbdd-c746c97f26c3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:58.668635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:58.669559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:58.669840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:58.670086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:58.681244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:58.682703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:10:58.701062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:58.702084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:58.703621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:10:58.703621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:10:58.703621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:10:58.703621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:58.703621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 646/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:10:58.710852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:58.823743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21e1bb-c6c8-4296-a691-87e506f838d5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:58.860468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:58.861120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:58.861400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:58.861679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:58.862408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:58.862718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:58.864128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:58.876529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 624/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:10:58.882474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:58.921468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:59.007354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e89d-21d2-4503-ad50-65c5e14a591f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:59.042861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:59.043728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:59.044010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:59.044277 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:59.051275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:59.052747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:10:59.067237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:10:59.068208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:10:59.069115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 647/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:10:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:10:59.074656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:59.140763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:59.184039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44bdaa-827b-48fa-a92a-57946d8edc43 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:59.219864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63553c2-2980-4b1f-8953-bb1461eedde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:59.220539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63553c2-2980-4b1f-8953-bb1461eedde3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:59.220824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63553c2-2980-4b1f-8953-bb1461eedde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:59.221137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63553c2-2980-4b1f-8953-bb1461eedde3 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:59.221885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63553c2-2980-4b1f-8953-bb1461eedde3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:10:59.222193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63553c2-2980-4b1f-8953-bb1461eedde3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:59.223683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63553c2-2980-4b1f-8953-bb1461eedde3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:59.236483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 625/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:10:59.242562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:10:59.338724 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:10:59.348339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3b2f6-61d9-4166-b7f6-c62da3d1dc02 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:10:59.384997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57979972-5926-4dc2-8138-753f5f199b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:10:59.385884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979972-5926-4dc2-8138-753f5f199b46 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:10:59.386165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979972-5926-4dc2-8138-753f5f199b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:10:59.386437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979972-5926-4dc2-8138-753f5f199b46 None admin] PATH_INFO: `/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/b93c07dd32f543a69e74599297cbe17a/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:10:59.388675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57979972-5926-4dc2-8138-753f5f199b46 None admin] RBAC: Authorizing `identity:create_grant(project_id=772001415dd543878fb9fd8e3fe599f3, user_id=b93c07dd32f543a69e74599297cbe17a, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:10:59.390124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57979972-5926-4dc2-8138-753f5f199b46 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:10:59.405514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 648/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 08:10:59 2022] PUT /identity/v3/projects/772001415dd543878fb9fd8e3fe599f3/users/b93c07dd32f543a69e74599297cbe17a/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:11:01.384518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01cf17-bc0c-4e55-acf6-0ee473c39755 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:01.385372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01cf17-bc0c-4e55-acf6-0ee473c39755 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:01.385372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01cf17-bc0c-4e55-acf6-0ee473c39755 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:01.387674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 626/1274] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:11:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:11:01.393048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c65c70-58b2-4823-837d-9ace5a6a5834 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:01.393486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c65c70-58b2-4823-837d-9ace5a6a5834 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:01.393576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c65c70-58b2-4823-837d-9ace5a6a5834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:01.414769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-77c65c70-58b2-4823-837d-9ace5a6a5834 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:11:01.415989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-77c65c70-58b2-4823-837d-9ace5a6a5834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:11:01.486697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 649/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:11:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:11:01.510762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01cf17-bc0c-4e55-acf6-0ee473c39755 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:01.628848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01cf17-bc0c-4e55-acf6-0ee473c39755 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:01.646593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:01.667182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492c1d7-db84-4be5-a746-2b3ccb93cc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:01.667996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c1d7-db84-4be5-a746-2b3ccb93cc11 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:01.668276 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c1d7-db84-4be5-a746-2b3ccb93cc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:01.668585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c1d7-db84-4be5-a746-2b3ccb93cc11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:01.678445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492c1d7-db84-4be5-a746-2b3ccb93cc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c492c1d7-db84-4be5-a746-2b3ccb93cc11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 08:11:01.686051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:11:01.687106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:01.688246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 627/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 08:11:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:11:01.693433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c65c70-58b2-4823-837d-9ace5a6a5834 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:01.781045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:01.801338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c65c70-58b2-4823-837d-9ace5a6a5834 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:01.841048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:01.841715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:01.841997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:01.842270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:01.843049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:01.843355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:01.844779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:01.853689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 650/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 08:11:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:01.860634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492c1d7-db84-4be5-a746-2b3ccb93cc11 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:01.976369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492c1d7-db84-4be5-a746-2b3ccb93cc11 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:02.011412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:02.012235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:02.012524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:02.012799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:02.027376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:02.028854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:11:02.046225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:11:02.047251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:11:02.048321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:11:02.048321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:11:02.048321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:11:02.048321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:02.048321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 628/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:11:02.053467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:02.162820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591df5d-fd98-4793-9ed2-82a5778fe5b6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:02.198292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:02.198943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:02.199258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:02.199526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:02.200262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:02.200568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:02.201953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:02.213664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 651/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:02.222999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:02.276539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:02.352885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c61d9f-9a5c-43ab-acee-d9207076c0e0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:02.389141 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:02.389959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:02.390231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:02.390492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:02.397935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:02.399428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 08:11:02.413629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:11:02.414682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:02.415674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 629/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:11:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:11:02.422429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:02.426481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:02.539420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d899a-5bfa-4566-b85e-72e940b10d02 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:02.575003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:02.575700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:02.575979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:02.576246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:02.577077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:02.577386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:02.578824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:02.595882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 652/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 08:11:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:02.602427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:02.703847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:02.714450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2271-7520-4d36-9dec-069148f389a5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:02.749067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a83de-2c00-4e06-a51a-384b7ed8db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:02.749727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a83de-2c00-4e06-a51a-384b7ed8db75 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:02.749995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a83de-2c00-4e06-a51a-384b7ed8db75 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:02.750274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a83de-2c00-4e06-a51a-384b7ed8db75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:02.832596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a83de-2c00-4e06-a51a-384b7ed8db75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:02.833990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a83de-2c00-4e06-a51a-384b7ed8db75 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:02.924627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:02.935241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 630/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 08:11:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 08:11:02.941038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:03.003078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:03.041888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f9848-d535-4acd-8798-a4d58b60f3b6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:03.077112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:03.077889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:03.078170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:03.078431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:03.089656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:03.091342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 08:11:03.110097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:11:03.111123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:11:03.112155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 08:11:03.112155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 08:11:03.112155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 08:11:03.112155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:11:03.112155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 653/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 08:11:03.117407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a83de-2c00-4e06-a51a-384b7ed8db75 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:03.228867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a83de-2c00-4e06-a51a-384b7ed8db75 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:03.264634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:03.265289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:03.265566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:03.265842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:03.266568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:03.266876 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:03.268284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:03.279758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 631/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 08:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:03.285538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:03.389056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df583a-ec0e-489e-aaab-f83d0a15f76f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:03.403570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:03.426087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:03.426869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:03.427142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:03.427436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:03.434480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:03.435959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 08:11:03.450392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:11:03.451456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jun 23 08:11:03.452940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 654/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 08:11:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:11:03.460763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:03.519908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:03.574900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc5dee-87ef-4cdf-ba60-83ee1b8a6413 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:03.616364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fa5fa-6c51-4b8b-8fb9-64a7bfe78889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:03.617011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa5fa-6c51-4b8b-8fb9-64a7bfe78889 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:03.617288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa5fa-6c51-4b8b-8fb9-64a7bfe78889 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:03.617558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa5fa-6c51-4b8b-8fb9-64a7bfe78889 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:03.618647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fa5fa-6c51-4b8b-8fb9-64a7bfe78889 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:03.618951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fa5fa-6c51-4b8b-8fb9-64a7bfe78889 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:03.620450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fa5fa-6c51-4b8b-8fb9-64a7bfe78889 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:03.632917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 632/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 08:11:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:03.639360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:03.741499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb157d-fd56-4137-9e38-18452b6565ef None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:03.776960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365e25b-f842-4311-9fce-ebf82107a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:03.777591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365e25b-f842-4311-9fce-ebf82107a3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:03.777918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365e25b-f842-4311-9fce-ebf82107a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:03.778189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365e25b-f842-4311-9fce-ebf82107a3f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:03.779002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365e25b-f842-4311-9fce-ebf82107a3f7 None admin] RBAC: Adding query filter params (role.id=26c4193120bd43c69b4c3756d41b3949, scope.project.id=772001415dd543878fb9fd8e3fe599f3, user.id=b93c07dd32f543a69e74599297cbe17a) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:03.779334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365e25b-f842-4311-9fce-ebf82107a3f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:03.780770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365e25b-f842-4311-9fce-ebf82107a3f7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:03.786662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 655/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 08:11:03 2022] GET /identity/v3/role_assignments?user.id=b93c07dd32f543a69e74599297cbe17a&scope.project.id=772001415dd543878fb9fd8e3fe599f3&role.id=26c4193120bd43c69b4c3756d41b3949 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:11:05.812167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aeac5-af83-4c3e-8128-5c6e92c991fe None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:05.812938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aeac5-af83-4c3e-8128-5c6e92c991fe None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:05.812938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aeac5-af83-4c3e-8128-5c6e92c991fe None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:05.814915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 633/1288] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:11:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:11:05.820291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2119672-2f1d-47b6-8fff-cbc847cb406f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:05.820745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2119672-2f1d-47b6-8fff-cbc847cb406f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:05.820847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2119672-2f1d-47b6-8fff-cbc847cb406f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:05.838643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f2119672-2f1d-47b6-8fff-cbc847cb406f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:11:05.839929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f2119672-2f1d-47b6-8fff-cbc847cb406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:11:05.873118 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:05.914023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 656/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:11:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:11:05.936970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977aeac5-af83-4c3e-8128-5c6e92c991fe None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:06.052213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977aeac5-af83-4c3e-8128-5c6e92c991fe None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:06.062990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:06.087612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:06.088411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:06.088692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:06.088964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:06.089611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:06.091411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:11:06.103247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:11:06.104416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:06.105583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 634/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:11:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:11:06.112884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2119672-2f1d-47b6-8fff-cbc847cb406f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:06.221288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2119672-2f1d-47b6-8fff-cbc847cb406f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:06.265319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d0338-a3e1-450a-809f-69771a684b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:06.265997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0338-a3e1-450a-809f-69771a684b0c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:06.266278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0338-a3e1-450a-809f-69771a684b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:06.266553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0338-a3e1-450a-809f-69771a684b0c None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:06.267300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d0338-a3e1-450a-809f-69771a684b0c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:06.267606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d0338-a3e1-450a-809f-69771a684b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:06.269001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d0338-a3e1-450a-809f-69771a684b0c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:06.275736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 657/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 23 08:11:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:06.282623 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:06.390040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b469cdf-4822-4944-8f75-9db7b1bd6fc6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:06.424823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1757edd-1fef-4f15-b1e0-3a7c53907019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:06.425480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757edd-1fef-4f15-b1e0-3a7c53907019 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:06.425767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757edd-1fef-4f15-b1e0-3a7c53907019 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:06.426040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757edd-1fef-4f15-b1e0-3a7c53907019 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:06.426745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1757edd-1fef-4f15-b1e0-3a7c53907019 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:06.427050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1757edd-1fef-4f15-b1e0-3a7c53907019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:06.428447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1757edd-1fef-4f15-b1e0-3a7c53907019 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:06.432949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 635/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 23 08:11:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:11:08.469315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ef16-803b-4fa9-8f22-f07da857910f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:08.470142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ef16-803b-4fa9-8f22-f07da857910f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:08.470142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ef16-803b-4fa9-8f22-f07da857910f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:08.471804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 658/1293] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:11:08.477763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3550b6-56da-419c-a9c9-682fbce5c38e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:08.478225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3550b6-56da-419c-a9c9-682fbce5c38e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:08.478344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3550b6-56da-419c-a9c9-682fbce5c38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:08.503843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fc3550b6-56da-419c-a9c9-682fbce5c38e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:11:08.505060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fc3550b6-56da-419c-a9c9-682fbce5c38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:11:08.542898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:08.585603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 636/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:11:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:11:08.610424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8ef16-803b-4fa9-8f22-f07da857910f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:08.650357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:08.729706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8ef16-803b-4fa9-8f22-f07da857910f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:08.764809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4fd77-86dd-4a7b-9ece-b61b5b85e576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:08.765624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4fd77-86dd-4a7b-9ece-b61b5b85e576 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:08.765754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4fd77-86dd-4a7b-9ece-b61b5b85e576 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:08.766042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4fd77-86dd-4a7b-9ece-b61b5b85e576 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:08.766795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4fd77-86dd-4a7b-9ece-b61b5b85e576 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:08.768158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4fd77-86dd-4a7b-9ece-b61b5b85e576 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:08.785609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 659/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 08:11:08 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:11:10.812739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2790f2-b2a4-43d2-8db0-ddf96d79fac0 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:10.813272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2790f2-b2a4-43d2-8db0-ddf96d79fac0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:10.813272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2790f2-b2a4-43d2-8db0-ddf96d79fac0 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:10.814565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 637/1296] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:11:10.820903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712837-e075-414d-80db-168c023de474 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:10.821360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712837-e075-414d-80db-168c023de474 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:10.821465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712837-e075-414d-80db-168c023de474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:10.841519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-79712837-e075-414d-80db-168c023de474 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:11:10.842709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-79712837-e075-414d-80db-168c023de474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:11:10.924593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 660/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:11:10 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:11:10.947914 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2790f2-b2a4-43d2-8db0-ddf96d79fac0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:11.058992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2790f2-b2a4-43d2-8db0-ddf96d79fac0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:11.093603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:11.094330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:11.094609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:11.094884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:11.095567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:11.096941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:11:11.109090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:11:11.110203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:11.111089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 638/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:11:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:11:11.118997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712837-e075-414d-80db-168c023de474 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:11.141176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:11.227996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712837-e075-414d-80db-168c023de474 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:11.264022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:11.264688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:11.265253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:11.265429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:11.266148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:11.266451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:11.267876 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:11.274246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 661/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 23 08:11:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:11.281282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:11.316028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:11.389740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769ecae-1f8b-4a58-bbd5-ed3fd86d38a4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:11.424444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715e0cb-0971-46ed-a89e-ac1fed6136af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:11.425082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715e0cb-0971-46ed-a89e-ac1fed6136af None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:11.425354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715e0cb-0971-46ed-a89e-ac1fed6136af None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:11.425609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715e0cb-0971-46ed-a89e-ac1fed6136af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:11.426389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715e0cb-0971-46ed-a89e-ac1fed6136af None admin] RBAC: Adding query filter params (interface=public, service_id=c0eb16390751474abe8ebae364fa945f, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:11.426694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715e0cb-0971-46ed-a89e-ac1fed6136af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:11.428111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715e0cb-0971-46ed-a89e-ac1fed6136af None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:11.433116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 639/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 08:11:11 2022] GET /identity/v3/endpoints?service_id=c0eb16390751474abe8ebae364fa945f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:11.439717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:11.538477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:11.559348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a19aa0-0c51-4467-8a35-5ae164108fc0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:11.600746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf6889-a6fb-446d-9aef-c271d3ed1ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:11.601735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf6889-a6fb-446d-9aef-c271d3ed1ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:11.602140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf6889-a6fb-446d-9aef-c271d3ed1ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:11.602531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf6889-a6fb-446d-9aef-c271d3ed1ce8 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:11.603667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf6889-a6fb-446d-9aef-c271d3ed1ce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:11.605841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf6889-a6fb-446d-9aef-c271d3ed1ce8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:11.613241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 662/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 08:11:11 2022] GET /identity/v3/services => generated 2191 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:11:13.622444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d77a32-dcf4-4370-b254-495b5818bad0 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:13.623062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d77a32-dcf4-4370-b254-495b5818bad0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:13.623062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d77a32-dcf4-4370-b254-495b5818bad0 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:13.624397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 640/1302] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:11:13.630312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0cedc-685c-4cbd-8eb8-951f14e7af5f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:13.630761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0cedc-685c-4cbd-8eb8-951f14e7af5f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:13.630858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0cedc-685c-4cbd-8eb8-951f14e7af5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:13.655737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-46e0cedc-685c-4cbd-8eb8-951f14e7af5f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:11:13.656928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-46e0cedc-685c-4cbd-8eb8-951f14e7af5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:11:13.728038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 663/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:11:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 08:11:13.752800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d77a32-dcf4-4370-b254-495b5818bad0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:13.856726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:13.869140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d77a32-dcf4-4370-b254-495b5818bad0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:13.926702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:13.928089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:13.928527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:13.928950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:13.929823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:13.931242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 08:11:13.944500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 08:11:13.945736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jun 23 08:11:13.946856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 641/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 08:11:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 08:11:13.952601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0cedc-685c-4cbd-8eb8-951f14e7af5f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:14.031628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:14.056599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0cedc-685c-4cbd-8eb8-951f14e7af5f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:14.091795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cd2ad-5df3-464e-b778-23ad6c0139bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:14.092456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd2ad-5df3-464e-b778-23ad6c0139bf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:14.092740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd2ad-5df3-464e-b778-23ad6c0139bf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:14.093009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd2ad-5df3-464e-b778-23ad6c0139bf None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:14.093711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cd2ad-5df3-464e-b778-23ad6c0139bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:11:14.094020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cd2ad-5df3-464e-b778-23ad6c0139bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:14.095428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cd2ad-5df3-464e-b778-23ad6c0139bf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:14.100671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 664/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 23 08:11:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:11:14.107598 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:11:14.225645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de0ab3-22ab-4649-a107-a15d7aa6e008 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:11:14.264826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663688fc-4f75-41e3-a459-42625bf7af55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:11:14.265508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663688fc-4f75-41e3-a459-42625bf7af55 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:11:14.265799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663688fc-4f75-41e3-a459-42625bf7af55 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:11:14.266107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663688fc-4f75-41e3-a459-42625bf7af55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:11:14.267041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663688fc-4f75-41e3-a459-42625bf7af55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:11:14.268669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663688fc-4f75-41e3-a459-42625bf7af55 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:11:14.292045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 642/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 08:11:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:11:24.097613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=30 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:11:24.286468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:17.106280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ecb8c-17f6-4385-a9b4-efa195a176cc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:17.107423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ecb8c-17f6-4385-a9b4-efa195a176cc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:17.107919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ecb8c-17f6-4385-a9b4-efa195a176cc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:17.109680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 665/1307] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:17.116144 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8ce1-95a2-4f9d-9d0f-65b2161f423b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:17.117091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8ce1-95a2-4f9d-9d0f-65b2161f423b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:17.117652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8ce1-95a2-4f9d-9d0f-65b2161f423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:17.209532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-96ae8ce1-95a2-4f9d-9d0f-65b2161f423b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:17.211106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-96ae8ce1-95a2-4f9d-9d0f-65b2161f423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:17.345386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 643/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:17.399377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e486728-b2a1-4759-b9fb-002ef8a7e576 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:17.400018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e486728-b2a1-4759-b9fb-002ef8a7e576 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:17.400522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e486728-b2a1-4759-b9fb-002ef8a7e576 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:17.402736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 666/1309] 10.222.0.33 () {54 vars in 964 bytes} [Thu Jun 23 08:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:17.409281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765c521-3062-40da-a4bd-3f427936a9cd None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:17.409581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765c521-3062-40da-a4bd-3f427936a9cd None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:17.409948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765c521-3062-40da-a4bd-3f427936a9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:17.498610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9765c521-3062-40da-a4bd-3f427936a9cd None None] MFA Rules not processed for user `2965b1c8ab52457f8f167acda4ba6844`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:17.500062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9765c521-3062-40da-a4bd-3f427936a9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:17.522670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:17.588878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 644/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Jun 23 08:12:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:17.601004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e486728-b2a1-4759-b9fb-002ef8a7e576 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:17.604301 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8e486728-b2a1-4759-b9fb-002ef8a7e576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:17.736428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e486728-b2a1-4759-b9fb-002ef8a7e576 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:17.840855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf93be-f8d4-440c-8606-8b9cdbbbc17a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:17.841534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf93be-f8d4-440c-8606-8b9cdbbbc17a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:17.841819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf93be-f8d4-440c-8606-8b9cdbbbc17a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:17.842093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf93be-f8d4-440c-8606-8b9cdbbbc17a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:17.843879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d3cf93be-f8d4-440c-8606-8b9cdbbbc17a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:17.905193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:17.968534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf93be-f8d4-440c-8606-8b9cdbbbc17a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:17.969895 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf93be-f8d4-440c-8606-8b9cdbbbc17a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:17.989798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 667/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:12:20.313298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfb78f-5ff8-4b14-8b09-78694f6340e3 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:20.313806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfb78f-5ff8-4b14-8b09-78694f6340e3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:20.313904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfb78f-5ff8-4b14-8b09-78694f6340e3 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:20.315269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 645/1312] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:20.321175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b178b4-14d6-4b22-ad55-cc9e06f5c30f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:20.321834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b178b4-14d6-4b22-ad55-cc9e06f5c30f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:20.322098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b178b4-14d6-4b22-ad55-cc9e06f5c30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:20.401608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a8b178b4-14d6-4b22-ad55-cc9e06f5c30f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:20.402820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a8b178b4-14d6-4b22-ad55-cc9e06f5c30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:20.514708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 668/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:20.552564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfb78f-5ff8-4b14-8b09-78694f6340e3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:20.554091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ffcfb78f-5ff8-4b14-8b09-78694f6340e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:20.675415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfb78f-5ff8-4b14-8b09-78694f6340e3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:20.757104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:20.760828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e185d-98da-4f0a-9e41-d1d4dcc7eb17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:20.761810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e185d-98da-4f0a-9e41-d1d4dcc7eb17 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:20.762084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e185d-98da-4f0a-9e41-d1d4dcc7eb17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:20.762314 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e185d-98da-4f0a-9e41-d1d4dcc7eb17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:20.764568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e08e185d-98da-4f0a-9e41-d1d4dcc7eb17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:20.880654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e185d-98da-4f0a-9e41-d1d4dcc7eb17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:20.882146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e185d-98da-4f0a-9e41-d1d4dcc7eb17 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:20.904130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 646/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:23.237868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ba7e-915e-4786-8276-c8a4e4fe45f1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:23.238698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ba7e-915e-4786-8276-c8a4e4fe45f1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:23.238698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ba7e-915e-4786-8276-c8a4e4fe45f1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:23.241389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 669/1315] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:23.246792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6c384-e38a-4616-b895-3701d2cb7b18 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:23.247258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6c384-e38a-4616-b895-3701d2cb7b18 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:23.247395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6c384-e38a-4616-b895-3701d2cb7b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:23.321517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2ec6c384-e38a-4616-b895-3701d2cb7b18 None None] MFA Rules not processed for user `9f5f4a174ee24efbbdabdbbd9715f21e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:23.322640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2ec6c384-e38a-4616-b895-3701d2cb7b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:23.381118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:23.435060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 647/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:23 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:23.477518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ba7e-915e-4786-8276-c8a4e4fe45f1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:23.479059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1743ba7e-915e-4786-8276-c8a4e4fe45f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:23.491724 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:23.592528 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ba7e-915e-4786-8276-c8a4e4fe45f1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:23.672367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f697c1d-39c1-4757-801e-188b7bb3650d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:23.673010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f697c1d-39c1-4757-801e-188b7bb3650d service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:23.673294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f697c1d-39c1-4757-801e-188b7bb3650d service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:23.673567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f697c1d-39c1-4757-801e-188b7bb3650d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:23.675235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6f697c1d-39c1-4757-801e-188b7bb3650d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:23.744076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:23.793154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f697c1d-39c1-4757-801e-188b7bb3650d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:23.794603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f697c1d-39c1-4757-801e-188b7bb3650d service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:23.813443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 670/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:26.406751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d60ff-f09b-4dc6-bbb2-7e54ae78d9d8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:26.407693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d60ff-f09b-4dc6-bbb2-7e54ae78d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:26.407693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d60ff-f09b-4dc6-bbb2-7e54ae78d9d8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:26.409857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 648/1318] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:26.415581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8f2b0-04bf-41db-8ce3-e2a33af3ff09 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:26.416009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8f2b0-04bf-41db-8ce3-e2a33af3ff09 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:26.416101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8f2b0-04bf-41db-8ce3-e2a33af3ff09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:26.491424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3db8f2b0-04bf-41db-8ce3-e2a33af3ff09 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:26.493109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3db8f2b0-04bf-41db-8ce3-e2a33af3ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:26.600042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 671/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:26.642129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d60ff-f09b-4dc6-bbb2-7e54ae78d9d8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:26.644587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-815d60ff-f09b-4dc6-bbb2-7e54ae78d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:26.755988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d60ff-f09b-4dc6-bbb2-7e54ae78d9d8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:26.831715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ceb0e-0200-4e34-858f-226055c285d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:26.832353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ceb0e-0200-4e34-858f-226055c285d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:26.832629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ceb0e-0200-4e34-858f-226055c285d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:26.832918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ceb0e-0200-4e34-858f-226055c285d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:26.834500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ee1ceb0e-0200-4e34-858f-226055c285d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:26.842148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:26.958723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ceb0e-0200-4e34-858f-226055c285d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:26.960264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ceb0e-0200-4e34-858f-226055c285d7 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:26.979239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 649/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:29.648446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c9725-dbb9-4072-bbe3-67542c1afe06 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:29.649098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c9725-dbb9-4072-bbe3-67542c1afe06 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:29.649098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c9725-dbb9-4072-bbe3-67542c1afe06 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:29.650345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 672/1321] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:29.655760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b3815-f4fe-43c1-b134-080b3608dae9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:29.656223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b3815-f4fe-43c1-b134-080b3608dae9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:29.656320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b3815-f4fe-43c1-b134-080b3608dae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:29.722825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-828b3815-f4fe-43c1-b134-080b3608dae9 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:29.724064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-828b3815-f4fe-43c1-b134-080b3608dae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:29.784126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:29.829571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 650/1322] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:29.878846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8c31b-3188-4dba-8dfb-c8e88abac440 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:29.879137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8c31b-3188-4dba-8dfb-c8e88abac440 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:29.879446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8c31b-3188-4dba-8dfb-c8e88abac440 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:29.880846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 673/1323] 10.222.0.33 () {54 vars in 964 bytes} [Thu Jun 23 08:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:29.887695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4989c-498c-4112-96dc-e4707d83cc70 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:29.887971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4989c-498c-4112-96dc-e4707d83cc70 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:29.888279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4989c-498c-4112-96dc-e4707d83cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:29.974795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6de4989c-498c-4112-96dc-e4707d83cc70 None None] MFA Rules not processed for user `2965b1c8ab52457f8f167acda4ba6844`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:29.976015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6de4989c-498c-4112-96dc-e4707d83cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:30.058672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 651/1324] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Jun 23 08:12:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:30.067859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8c31b-3188-4dba-8dfb-c8e88abac440 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:30.069453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ede8c31b-3188-4dba-8dfb-c8e88abac440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:30.089531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:30.178095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8c31b-3188-4dba-8dfb-c8e88abac440 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:30.255851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef446835-f50d-4202-92b9-c27b9403059a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:30.256397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef446835-f50d-4202-92b9-c27b9403059a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:30.256676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef446835-f50d-4202-92b9-c27b9403059a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:30.256948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef446835-f50d-4202-92b9-c27b9403059a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:30.258748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef446835-f50d-4202-92b9-c27b9403059a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:30.347365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:30.390066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef446835-f50d-4202-92b9-c27b9403059a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:30.391569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef446835-f50d-4202-92b9-c27b9403059a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:30.409675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 674/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:32.688784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b683ea-8fbc-4d73-8407-e492c41b0bb8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:32.689396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b683ea-8fbc-4d73-8407-e492c41b0bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:32.689396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b683ea-8fbc-4d73-8407-e492c41b0bb8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:32.690731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 652/1326] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:32.696235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ee5f8-7afe-4963-85fe-1ff1707a44b8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:32.696722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ee5f8-7afe-4963-85fe-1ff1707a44b8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:32.696860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ee5f8-7afe-4963-85fe-1ff1707a44b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:32.762780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c11ee5f8-7afe-4963-85fe-1ff1707a44b8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:32.763935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c11ee5f8-7afe-4963-85fe-1ff1707a44b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:32.874033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 675/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:32.909983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b683ea-8fbc-4d73-8407-e492c41b0bb8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:32.911559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-13b683ea-8fbc-4d73-8407-e492c41b0bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:32.982285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:33.029449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b683ea-8fbc-4d73-8407-e492c41b0bb8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:33.106292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a6363-ac86-47ea-aa93-046d76ffe90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:33.106935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a6363-ac86-47ea-aa93-046d76ffe90a service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:33.107237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a6363-ac86-47ea-aa93-046d76ffe90a service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:33.107523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a6363-ac86-47ea-aa93-046d76ffe90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:33.109162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-903a6363-ac86-47ea-aa93-046d76ffe90a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:33.218537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:33.221398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a6363-ac86-47ea-aa93-046d76ffe90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:33.223536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a6363-ac86-47ea-aa93-046d76ffe90a service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:33.243810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 653/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:35.316190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e616e-0d05-4ee4-9289-92434798fb43 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:35.316731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e616e-0d05-4ee4-9289-92434798fb43 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:35.316731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e616e-0d05-4ee4-9289-92434798fb43 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:35.318108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 676/1329] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:35.323861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd706823-a9f8-4278-a556-f042ffc94c92 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:35.324279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd706823-a9f8-4278-a556-f042ffc94c92 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:35.324533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd706823-a9f8-4278-a556-f042ffc94c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:35.409528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-dd706823-a9f8-4278-a556-f042ffc94c92 None None] MFA Rules not processed for user `9f5f4a174ee24efbbdabdbbd9715f21e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:35.410767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dd706823-a9f8-4278-a556-f042ffc94c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:35.511307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 654/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:35.550904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e616e-0d05-4ee4-9289-92434798fb43 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:35.552483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-647e616e-0d05-4ee4-9289-92434798fb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:35.586600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:35.664005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e616e-0d05-4ee4-9289-92434798fb43 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:35.745488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7912042-ebee-4032-8824-f30b021808e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:35.746148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7912042-ebee-4032-8824-f30b021808e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:35.746428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7912042-ebee-4032-8824-f30b021808e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:35.746700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7912042-ebee-4032-8824-f30b021808e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:35.748521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f7912042-ebee-4032-8824-f30b021808e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:35.829976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:35.863033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7912042-ebee-4032-8824-f30b021808e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:35.864526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7912042-ebee-4032-8824-f30b021808e9 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:35.883384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 677/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:38.312431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6167d7-9194-4fd5-8086-fb37793912f2 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:38.313271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6167d7-9194-4fd5-8086-fb37793912f2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:38.313271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6167d7-9194-4fd5-8086-fb37793912f2 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:38.315525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 655/1332] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:38.321811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb6191-21cd-415a-8270-c00a900330b5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:38.322523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb6191-21cd-415a-8270-c00a900330b5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:38.322665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb6191-21cd-415a-8270-c00a900330b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:38.406806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5abb6191-21cd-415a-8270-c00a900330b5 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:38.408054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5abb6191-21cd-415a-8270-c00a900330b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:38.517024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 678/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:38.555977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6167d7-9194-4fd5-8086-fb37793912f2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:38.558440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6e6167d7-9194-4fd5-8086-fb37793912f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:38.638618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:38.680025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6167d7-9194-4fd5-8086-fb37793912f2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:38.758673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c14a9c-b5ae-4b05-a055-2f90db2de714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:38.759359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c14a9c-b5ae-4b05-a055-2f90db2de714 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:38.759647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c14a9c-b5ae-4b05-a055-2f90db2de714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:38.759894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c14a9c-b5ae-4b05-a055-2f90db2de714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:38.761599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e5c14a9c-b5ae-4b05-a055-2f90db2de714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:38.892740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c14a9c-b5ae-4b05-a055-2f90db2de714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:38.894100 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c14a9c-b5ae-4b05-a055-2f90db2de714 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:38.904808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:38.917215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 656/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:40.982734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c75fd-b118-4196-be09-d479a6ecc0b2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:40.983331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c75fd-b118-4196-be09-d479a6ecc0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:40.983331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c75fd-b118-4196-be09-d479a6ecc0b2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:40.985459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 679/1335] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:40.993873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5d1e6-5b16-4a11-870a-084412e6de57 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:40.994329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5d1e6-5b16-4a11-870a-084412e6de57 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:40.994441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5d1e6-5b16-4a11-870a-084412e6de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:41.076921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-06f5d1e6-5b16-4a11-870a-084412e6de57 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:41.078762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-06f5d1e6-5b16-4a11-870a-084412e6de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:41.189763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 657/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:41.229896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c75fd-b118-4196-be09-d479a6ecc0b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:41.231465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-218c75fd-b118-4196-be09-d479a6ecc0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:41.273940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:41.339922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c75fd-b118-4196-be09-d479a6ecc0b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:41.444911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ba6e4-9d0f-4704-a8e9-ab4c0bb92a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:41.445555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ba6e4-9d0f-4704-a8e9-ab4c0bb92a18 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:41.445833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ba6e4-9d0f-4704-a8e9-ab4c0bb92a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:41.446106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ba6e4-9d0f-4704-a8e9-ab4c0bb92a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:41.447787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e90ba6e4-9d0f-4704-a8e9-ab4c0bb92a18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:41.541298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:41.569908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ba6e4-9d0f-4704-a8e9-ab4c0bb92a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:41.571610 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ba6e4-9d0f-4704-a8e9-ab4c0bb92a18 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:41.590627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 680/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:43.922843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22bd67-dd99-40de-9685-f6887c2bfc73 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:43.923756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22bd67-dd99-40de-9685-f6887c2bfc73 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:43.923756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22bd67-dd99-40de-9685-f6887c2bfc73 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:43.925951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 658/1338] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:43.935458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80f20f-6027-4750-ab33-1cf2455a4db8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:43.935878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80f20f-6027-4750-ab33-1cf2455a4db8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:43.936044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80f20f-6027-4750-ab33-1cf2455a4db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:44.012620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0e80f20f-6027-4750-ab33-1cf2455a4db8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:44.013827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0e80f20f-6027-4750-ab33-1cf2455a4db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:44.117149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 681/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:44.153091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22bd67-dd99-40de-9685-f6887c2bfc73 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:44.154642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0b22bd67-dd99-40de-9685-f6887c2bfc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:44.243374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:44.272182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22bd67-dd99-40de-9685-f6887c2bfc73 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:44.351706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08d721-df6f-457a-b2e8-9e290554fa34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:44.352357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08d721-df6f-457a-b2e8-9e290554fa34 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:44.352640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08d721-df6f-457a-b2e8-9e290554fa34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:44.352911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08d721-df6f-457a-b2e8-9e290554fa34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:44.354566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dd08d721-df6f-457a-b2e8-9e290554fa34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:44.470584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08d721-df6f-457a-b2e8-9e290554fa34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:44.472042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08d721-df6f-457a-b2e8-9e290554fa34 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:44.494953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 659/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:48.650048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb582-0f9a-4929-baa7-cba392cd7e3d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:48.650628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb582-0f9a-4929-baa7-cba392cd7e3d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:48.650628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb582-0f9a-4929-baa7-cba392cd7e3d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:48.655722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 682/1341] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:48.658487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4ece1-1c02-4252-a3bb-28e96f4e3b32 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:48.658977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4ece1-1c02-4252-a3bb-28e96f4e3b32 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:48.659075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4ece1-1c02-4252-a3bb-28e96f4e3b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:48.666554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:48.734219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b7c4ece1-1c02-4252-a3bb-28e96f4e3b32 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:48.735585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b7c4ece1-1c02-4252-a3bb-28e96f4e3b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:48.843227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 660/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:48.882704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb582-0f9a-4929-baa7-cba392cd7e3d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:48.885183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-707cb582-0f9a-4929-baa7-cba392cd7e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:48.939252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:12:48.987504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb582-0f9a-4929-baa7-cba392cd7e3d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:48.992221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f777e-f74a-4b27-a28b-e83e7e87ad21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:48.992871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f777e-f74a-4b27-a28b-e83e7e87ad21 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:48.993155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f777e-f74a-4b27-a28b-e83e7e87ad21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:48.993434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f777e-f74a-4b27-a28b-e83e7e87ad21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:49.002700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f777e-f74a-4b27-a28b-e83e7e87ad21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:49.004189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f777e-f74a-4b27-a28b-e83e7e87ad21 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:49.013869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 683/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:51.843735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f72b8-b973-442b-a7d4-6dc6585550e1 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:51.844300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f72b8-b973-442b-a7d4-6dc6585550e1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:51.844300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f72b8-b973-442b-a7d4-6dc6585550e1 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:51.845631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 661/1344] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:51.851619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7dc20-ebc2-446f-9bf1-fab830364c09 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:51.852207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7dc20-ebc2-446f-9bf1-fab830364c09 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:51.852207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7dc20-ebc2-446f-9bf1-fab830364c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:51.882756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6be7dc20-ebc2-446f-9bf1-fab830364c09 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:51.884030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6be7dc20-ebc2-446f-9bf1-fab830364c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:51.969981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 684/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:52.008014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f72b8-b973-442b-a7d4-6dc6585550e1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:52.009948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bc3f72b8-b973-442b-a7d4-6dc6585550e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:52.023955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f72b8-b973-442b-a7d4-6dc6585550e1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:52.027589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555b8b9-6c4c-4e72-bea9-bf85b61b6b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:52.028239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555b8b9-6c4c-4e72-bea9-bf85b61b6b50 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:52.028517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555b8b9-6c4c-4e72-bea9-bf85b61b6b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:52.028791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555b8b9-6c4c-4e72-bea9-bf85b61b6b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:52.038216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555b8b9-6c4c-4e72-bea9-bf85b61b6b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:52.039679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555b8b9-6c4c-4e72-bea9-bf85b61b6b50 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:52.049358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 662/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:12:56.112242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cd275-45a4-408f-b16d-d3417294db50 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:56.112734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cd275-45a4-408f-b16d-d3417294db50 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:56.112887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cd275-45a4-408f-b16d-d3417294db50 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:56.114604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 685/1347] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:12:56.120334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0972e8e-dc00-4227-916f-33479709d99f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:56.120870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0972e8e-dc00-4227-916f-33479709d99f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:56.120870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0972e8e-dc00-4227-916f-33479709d99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:56.140138 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d0972e8e-dc00-4227-916f-33479709d99f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:12:56.141860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d0972e8e-dc00-4227-916f-33479709d99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:12:56.154103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 663/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:12:56.198192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cd275-45a4-408f-b16d-d3417294db50 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:12:56.210915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cd275-45a4-408f-b16d-d3417294db50 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:12:56.216131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a2054-1b78-4b90-ae5b-6bc428dc97b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:12:56.216830 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2054-1b78-4b90-ae5b-6bc428dc97b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:12:56.217299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2054-1b78-4b90-ae5b-6bc428dc97b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:12:56.217420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2054-1b78-4b90-ae5b-6bc428dc97b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:12:56.228579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a2054-1b78-4b90-ae5b-6bc428dc97b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:12:56.230070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a2054-1b78-4b90-ae5b-6bc428dc97b3 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:12:56.241122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 686/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:01.220915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb550b-1d60-44b6-bb97-35e65f769c64 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:01.220915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb550b-1d60-44b6-bb97-35e65f769c64 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:01.221511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb550b-1d60-44b6-bb97-35e65f769c64 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:01.223171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 664/1350] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:01.229792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e575a9-0d83-4722-bd3e-0a078e17e6af None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:01.230288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e575a9-0d83-4722-bd3e-0a078e17e6af None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:01.230555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e575a9-0d83-4722-bd3e-0a078e17e6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:01.309051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-26e575a9-0d83-4722-bd3e-0a078e17e6af None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:01.310349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-26e575a9-0d83-4722-bd3e-0a078e17e6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:01.340997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:13:01.409924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 687/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:13:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:05.580215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1928e-b3a8-445f-958d-39d8930690c2 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:05.580819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1928e-b3a8-445f-958d-39d8930690c2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:05.580819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1928e-b3a8-445f-958d-39d8930690c2 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:05.582270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 665/1352] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:05.587252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1e3c-bd21-4c81-8af9-13e073eeab45 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:05.588085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1e3c-bd21-4c81-8af9-13e073eeab45 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:05.588594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1e3c-bd21-4c81-8af9-13e073eeab45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:05.608173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cf5a1e3c-bd21-4c81-8af9-13e073eeab45 None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:05.609496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cf5a1e3c-bd21-4c81-8af9-13e073eeab45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:05.618763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 688/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:13:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:06.134140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=72 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:13:07.027022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7df3c-801e-4874-a67d-7f59a16b2887 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:07.027594 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7df3c-801e-4874-a67d-7f59a16b2887 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:07.027690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7df3c-801e-4874-a67d-7f59a16b2887 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:07.029346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 666/1354] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:07.034270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7277d-7a49-4c03-a04c-a8ca2c434cea None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:07.034674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7277d-7a49-4c03-a04c-a8ca2c434cea None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:07.034970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7277d-7a49-4c03-a04c-a8ca2c434cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:07.056045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fed7277d-7a49-4c03-a04c-a8ca2c434cea None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:07.057266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fed7277d-7a49-4c03-a04c-a8ca2c434cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:07.066730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 689/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:13:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:10.687432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9949f-17f2-4cac-a1f3-fafa1c0ee523 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:10.687973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9949f-17f2-4cac-a1f3-fafa1c0ee523 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:10.687973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9949f-17f2-4cac-a1f3-fafa1c0ee523 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:10.690863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 667/1356] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:13:10.694621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9a855-bd9e-4916-99ba-151b827754e2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:10.695456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9a855-bd9e-4916-99ba-151b827754e2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:10.695899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9a855-bd9e-4916-99ba-151b827754e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:10.718474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4fc9a855-bd9e-4916-99ba-151b827754e2 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:10.719906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4fc9a855-bd9e-4916-99ba-151b827754e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:10.729019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 690/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:10.787546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda8b1-5e05-4e78-a7b7-0628d585437d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:10.788130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda8b1-5e05-4e78-a7b7-0628d585437d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:10.788594 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda8b1-5e05-4e78-a7b7-0628d585437d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:10.790565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 668/1358] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 23 08:13:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:10.798621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc191c33-f1b9-4c55-843b-4acc74979c9b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:10.799181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc191c33-f1b9-4c55-843b-4acc74979c9b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:10.799684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc191c33-f1b9-4c55-843b-4acc74979c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:10.852156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc191c33-f1b9-4c55-843b-4acc74979c9b None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:10.853335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc191c33-f1b9-4c55-843b-4acc74979c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:10.939492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 691/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 23 08:13:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:13:10.948068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda8b1-5e05-4e78-a7b7-0628d585437d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:10.963587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda8b1-5e05-4e78-a7b7-0628d585437d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:10.969073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d1bfb-d9a1-4efe-a046-35fe1a4c56a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:10.969717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d1bfb-d9a1-4efe-a046-35fe1a4c56a5 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:10.970108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d1bfb-d9a1-4efe-a046-35fe1a4c56a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:10.970390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d1bfb-d9a1-4efe-a046-35fe1a4c56a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:10.979602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d1bfb-d9a1-4efe-a046-35fe1a4c56a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:10.981365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d1bfb-d9a1-4efe-a046-35fe1a4c56a5 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:10.992047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 669/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:13.214377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622144f-065a-484e-80c7-0a0b9ec8ad97 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:13.214890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622144f-065a-484e-80c7-0a0b9ec8ad97 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:13.214998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622144f-065a-484e-80c7-0a0b9ec8ad97 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:13.216654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 692/1361] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:13:13.222053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7139f5-4001-4d0c-9e21-52efc7271fe0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:13.222571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7139f5-4001-4d0c-9e21-52efc7271fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:13.222571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7139f5-4001-4d0c-9e21-52efc7271fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:13.242443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bf7139f5-4001-4d0c-9e21-52efc7271fe0 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:13.243656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bf7139f5-4001-4d0c-9e21-52efc7271fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:13.252549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 670/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:13.297106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622144f-065a-484e-80c7-0a0b9ec8ad97 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:13.311240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622144f-065a-484e-80c7-0a0b9ec8ad97 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:13.316496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cff14-b175-480c-98bb-9618014b7381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:13.317144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cff14-b175-480c-98bb-9618014b7381 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:13.317422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cff14-b175-480c-98bb-9618014b7381 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:13.317793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cff14-b175-480c-98bb-9618014b7381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:13.326611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cff14-b175-480c-98bb-9618014b7381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:13.328238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cff14-b175-480c-98bb-9618014b7381 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:13.338481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 693/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:15.673777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20083a8-c070-40e7-b05e-c8094ee36e31 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:15.674289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20083a8-c070-40e7-b05e-c8094ee36e31 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:15.674827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20083a8-c070-40e7-b05e-c8094ee36e31 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:15.675789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 671/1364] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:15.681405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8912d-99d1-4ca9-9d2d-ed28cde9ffae None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:15.681900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8912d-99d1-4ca9-9d2d-ed28cde9ffae None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:15.681998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8912d-99d1-4ca9-9d2d-ed28cde9ffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:15.700399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dda8912d-99d1-4ca9-9d2d-ed28cde9ffae None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:15.701590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dda8912d-99d1-4ca9-9d2d-ed28cde9ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:15.709887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 694/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:15.765906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a043d-38b9-44cf-8336-b1ba781b94f7 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:15.766404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a043d-38b9-44cf-8336-b1ba781b94f7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:15.766864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a043d-38b9-44cf-8336-b1ba781b94f7 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:15.770888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 672/1366] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 23 08:13:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:15.775117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9a4bf-a519-4b6f-8115-6060d9653650 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:15.775639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9a4bf-a519-4b6f-8115-6060d9653650 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:15.776063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9a4bf-a519-4b6f-8115-6060d9653650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:15.796045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-05a9a4bf-a519-4b6f-8115-6060d9653650 None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:15.797147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-05a9a4bf-a519-4b6f-8115-6060d9653650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:15.806473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 695/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 23 08:13:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:15.814677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a043d-38b9-44cf-8336-b1ba781b94f7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:15.829352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a043d-38b9-44cf-8336-b1ba781b94f7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:15.835103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479f33f-3e3a-40bf-a512-94f3ae98a587 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:15.835785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479f33f-3e3a-40bf-a512-94f3ae98a587 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:15.836069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479f33f-3e3a-40bf-a512-94f3ae98a587 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:15.836340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479f33f-3e3a-40bf-a512-94f3ae98a587 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:15.844453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479f33f-3e3a-40bf-a512-94f3ae98a587 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:15.846037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479f33f-3e3a-40bf-a512-94f3ae98a587 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:15.855717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 673/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:17.962421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d6f9e-c6eb-4dda-9259-78082164ac9b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:17.963033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d6f9e-c6eb-4dda-9259-78082164ac9b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:17.963033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d6f9e-c6eb-4dda-9259-78082164ac9b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:17.964533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 696/1369] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:17.969968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2011f-3821-456b-9087-a71023e581c1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:17.970432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2011f-3821-456b-9087-a71023e581c1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:17.970521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2011f-3821-456b-9087-a71023e581c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:17.989106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a7e2011f-3821-456b-9087-a71023e581c1 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:17.990490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a7e2011f-3821-456b-9087-a71023e581c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:17.998874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 674/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:18.043226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d6f9e-c6eb-4dda-9259-78082164ac9b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:18.059454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d6f9e-c6eb-4dda-9259-78082164ac9b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:18.065451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22158ef6-ced1-450e-9ea8-c70de49e039e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:18.066523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22158ef6-ced1-450e-9ea8-c70de49e039e service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:18.067124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22158ef6-ced1-450e-9ea8-c70de49e039e service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:18.067701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22158ef6-ced1-450e-9ea8-c70de49e039e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:18.078193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22158ef6-ced1-450e-9ea8-c70de49e039e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:18.080619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22158ef6-ced1-450e-9ea8-c70de49e039e service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:18.093425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 697/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:18.748592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fdbd4-0e2a-4c8e-ac6e-635503b7cec4 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:18.749669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fdbd4-0e2a-4c8e-ac6e-635503b7cec4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:18.750011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fdbd4-0e2a-4c8e-ac6e-635503b7cec4 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:18.751564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 675/1372] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:13:18.757234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70950-2e00-4f38-b385-1c3745645639 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:18.757599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70950-2e00-4f38-b385-1c3745645639 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:18.757923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70950-2e00-4f38-b385-1c3745645639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:18.776103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-61d70950-2e00-4f38-b385-1c3745645639 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:18.777295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-61d70950-2e00-4f38-b385-1c3745645639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:18.785320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 698/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:19.121449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316714-84c0-475c-8add-3b83e94524f9 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:19.121449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316714-84c0-475c-8add-3b83e94524f9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:19.121449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316714-84c0-475c-8add-3b83e94524f9 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:19.124747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 676/1374] 10.222.0.33 () {54 vars in 963 bytes} [Thu Jun 23 08:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:19.127538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc963a6-c6dc-4ba8-96c2-0abc6ef337d2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:19.128072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc963a6-c6dc-4ba8-96c2-0abc6ef337d2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:19.128072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc963a6-c6dc-4ba8-96c2-0abc6ef337d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:19.135123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:13:19.181128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fdc963a6-c6dc-4ba8-96c2-0abc6ef337d2 None None] MFA Rules not processed for user `419321fbd62c4d95a39cfd498b4a97d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:19.182366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fdc963a6-c6dc-4ba8-96c2-0abc6ef337d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:19.259433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 699/1375] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Jun 23 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:19.262639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa316714-84c0-475c-8add-3b83e94524f9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:19.274096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa316714-84c0-475c-8add-3b83e94524f9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:19.277858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212269f-55ba-432b-a9ab-fc1d9a473107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:19.278684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212269f-55ba-432b-a9ab-fc1d9a473107 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:19.279126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212269f-55ba-432b-a9ab-fc1d9a473107 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:19.279608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212269f-55ba-432b-a9ab-fc1d9a473107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:19.289774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212269f-55ba-432b-a9ab-fc1d9a473107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:19.291417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212269f-55ba-432b-a9ab-fc1d9a473107 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:19.303288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 677/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:20.169314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a337d-94d6-4d83-8d75-76f876c6f8f6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:20.170466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a337d-94d6-4d83-8d75-76f876c6f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:20.170466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a337d-94d6-4d83-8d75-76f876c6f8f6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:20.180234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 700/1377] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:20.181450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92921de4-6266-4674-9f80-f6c0acf8086b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:20.181450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92921de4-6266-4674-9f80-f6c0acf8086b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:20.181450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92921de4-6266-4674-9f80-f6c0acf8086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:20.197624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-92921de4-6266-4674-9f80-f6c0acf8086b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:20.198840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-92921de4-6266-4674-9f80-f6c0acf8086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:20.207021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 678/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:20.252183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a337d-94d6-4d83-8d75-76f876c6f8f6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:20.267791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a337d-94d6-4d83-8d75-76f876c6f8f6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:20.273794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040e6d2-6e2f-4686-91e8-3c4517c5ecb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:20.274468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e6d2-6e2f-4686-91e8-3c4517c5ecb2 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:20.274771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e6d2-6e2f-4686-91e8-3c4517c5ecb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:20.275073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e6d2-6e2f-4686-91e8-3c4517c5ecb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:20.283514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040e6d2-6e2f-4686-91e8-3c4517c5ecb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:20.285111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040e6d2-6e2f-4686-91e8-3c4517c5ecb2 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:20.294629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 701/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:22.341265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978f101-8cfa-46cd-83ba-62321384b9f6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:22.341771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978f101-8cfa-46cd-83ba-62321384b9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:22.341771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978f101-8cfa-46cd-83ba-62321384b9f6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:22.343101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 679/1380] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:22.348670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b24a1d-99de-4108-9a9a-754fc7dfd602 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:22.349187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b24a1d-99de-4108-9a9a-754fc7dfd602 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:22.349187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b24a1d-99de-4108-9a9a-754fc7dfd602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:22.368781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b8b24a1d-99de-4108-9a9a-754fc7dfd602 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:22.369974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b8b24a1d-99de-4108-9a9a-754fc7dfd602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:22.378391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 702/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:22.423484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978f101-8cfa-46cd-83ba-62321384b9f6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:22.438364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978f101-8cfa-46cd-83ba-62321384b9f6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:22.444143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496fea5-610a-4f7c-8ac9-182dcf66ae43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:22.444786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496fea5-610a-4f7c-8ac9-182dcf66ae43 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:22.445886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496fea5-610a-4f7c-8ac9-182dcf66ae43 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:22.446336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496fea5-610a-4f7c-8ac9-182dcf66ae43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:22.455165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496fea5-610a-4f7c-8ac9-182dcf66ae43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:22.456771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496fea5-610a-4f7c-8ac9-182dcf66ae43 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:22.467599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 680/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:24.586066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc49a2-77f5-49d5-a8a4-ab7563669872 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:24.586635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc49a2-77f5-49d5-a8a4-ab7563669872 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:24.586635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc49a2-77f5-49d5-a8a4-ab7563669872 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:24.588795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 703/1383] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:24.593533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5be6-1bc8-41ee-bd95-971effd5b734 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:24.593984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5be6-1bc8-41ee-bd95-971effd5b734 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:24.594078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5be6-1bc8-41ee-bd95-971effd5b734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:24.612922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8a2b5be6-1bc8-41ee-bd95-971effd5b734 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:24.615066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8a2b5be6-1bc8-41ee-bd95-971effd5b734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:24.623819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 681/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:24.665356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc49a2-77f5-49d5-a8a4-ab7563669872 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:24.681575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc49a2-77f5-49d5-a8a4-ab7563669872 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:24.687469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3063189-033d-4336-a7f9-3dea99694904 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:24.688116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3063189-033d-4336-a7f9-3dea99694904 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:24.688392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3063189-033d-4336-a7f9-3dea99694904 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:24.688645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3063189-033d-4336-a7f9-3dea99694904 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:24.698766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3063189-033d-4336-a7f9-3dea99694904 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:24.700392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3063189-033d-4336-a7f9-3dea99694904 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:24.715386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 704/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:26.758205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b0936-291f-47da-b12e-c4149f14408c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:26.758653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b0936-291f-47da-b12e-c4149f14408c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:26.759060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b0936-291f-47da-b12e-c4149f14408c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:26.764087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 682/1386] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:26.765701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f57e59-08ee-43a4-8fe9-bdb98a8a6277 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:26.766112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f57e59-08ee-43a4-8fe9-bdb98a8a6277 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:26.767054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f57e59-08ee-43a4-8fe9-bdb98a8a6277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:26.796644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a5f57e59-08ee-43a4-8fe9-bdb98a8a6277 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:26.797842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a5f57e59-08ee-43a4-8fe9-bdb98a8a6277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:26.806059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 705/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:26.851808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b0936-291f-47da-b12e-c4149f14408c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:26.867280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b0936-291f-47da-b12e-c4149f14408c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:26.873560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b4aee-d4b1-44b8-babb-24d0f6a27b88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:26.874272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b4aee-d4b1-44b8-babb-24d0f6a27b88 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:26.874540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b4aee-d4b1-44b8-babb-24d0f6a27b88 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:26.874821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b4aee-d4b1-44b8-babb-24d0f6a27b88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:26.884644 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b4aee-d4b1-44b8-babb-24d0f6a27b88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:26.886378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b4aee-d4b1-44b8-babb-24d0f6a27b88 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:26.896880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 683/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:28.960439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14728f-7051-49c2-a135-b9a241a078c3 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:28.961086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14728f-7051-49c2-a135-b9a241a078c3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:28.961086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14728f-7051-49c2-a135-b9a241a078c3 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:28.963133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 706/1389] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:28.970059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41692ea4-e536-44c6-bb29-1c5190b28c08 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:28.970639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41692ea4-e536-44c6-bb29-1c5190b28c08 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:28.970639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41692ea4-e536-44c6-bb29-1c5190b28c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:28.994439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-41692ea4-e536-44c6-bb29-1c5190b28c08 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:28.996119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-41692ea4-e536-44c6-bb29-1c5190b28c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:29.007705 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 684/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:29.052439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14728f-7051-49c2-a135-b9a241a078c3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:29.068989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14728f-7051-49c2-a135-b9a241a078c3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:29.074626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed18ff-510b-40ef-ad86-22e1812f25ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:29.075326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed18ff-510b-40ef-ad86-22e1812f25ed service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:29.075613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed18ff-510b-40ef-ad86-22e1812f25ed service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:29.076193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed18ff-510b-40ef-ad86-22e1812f25ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:29.086129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed18ff-510b-40ef-ad86-22e1812f25ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:29.087807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed18ff-510b-40ef-ad86-22e1812f25ed service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:29.098503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 707/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:31.187931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7c521-a2a9-4e0f-8f0b-3a7d07223ed8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:31.188533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7c521-a2a9-4e0f-8f0b-3a7d07223ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:31.188533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7c521-a2a9-4e0f-8f0b-3a7d07223ed8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:31.189832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 685/1392] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:31.197532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c34cb-003a-46d2-9203-e7a873353ba6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:31.198002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c34cb-003a-46d2-9203-e7a873353ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:31.198255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c34cb-003a-46d2-9203-e7a873353ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:31.222330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6d6c34cb-003a-46d2-9203-e7a873353ba6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:31.224035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6d6c34cb-003a-46d2-9203-e7a873353ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:31.233728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 708/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:31.282669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7c521-a2a9-4e0f-8f0b-3a7d07223ed8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:31.297479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7c521-a2a9-4e0f-8f0b-3a7d07223ed8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:31.302993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee7b7-08f5-423b-998b-40f4a3f72405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:31.303696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee7b7-08f5-423b-998b-40f4a3f72405 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:31.303981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee7b7-08f5-423b-998b-40f4a3f72405 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:31.304261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee7b7-08f5-423b-998b-40f4a3f72405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:31.313043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee7b7-08f5-423b-998b-40f4a3f72405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:31.314763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee7b7-08f5-423b-998b-40f4a3f72405 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:31.325090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 686/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:33.402497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4877-dee6-47f6-af73-211073795ba8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:33.403072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4877-dee6-47f6-af73-211073795ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:33.403072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4877-dee6-47f6-af73-211073795ba8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:33.404477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 709/1395] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:33.409735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79d6a0-6f6b-4df2-99fe-e125d4079460 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:33.410189 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79d6a0-6f6b-4df2-99fe-e125d4079460 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:33.410312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79d6a0-6f6b-4df2-99fe-e125d4079460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:33.428551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9c79d6a0-6f6b-4df2-99fe-e125d4079460 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:33.429840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9c79d6a0-6f6b-4df2-99fe-e125d4079460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:33.438482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 687/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:33.483176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e4877-dee6-47f6-af73-211073795ba8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:33.500617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e4877-dee6-47f6-af73-211073795ba8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:33.506739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b1538-8966-46ff-8a7d-3c1b766715c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:33.507430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1538-8966-46ff-8a7d-3c1b766715c1 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:33.507716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1538-8966-46ff-8a7d-3c1b766715c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:33.507992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1538-8966-46ff-8a7d-3c1b766715c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:33.517200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b1538-8966-46ff-8a7d-3c1b766715c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:33.518804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b1538-8966-46ff-8a7d-3c1b766715c1 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:33.532941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 710/1397] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:34.472287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35bce-0236-4fc2-ba9a-070131e5d83d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:34.473303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35bce-0236-4fc2-ba9a-070131e5d83d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:34.473303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35bce-0236-4fc2-ba9a-070131e5d83d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:34.474245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 688/1398] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:34.479664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d41907-ceab-4c3c-8ba5-b99bdcb8e158 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:34.480135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d41907-ceab-4c3c-8ba5-b99bdcb8e158 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:34.480244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d41907-ceab-4c3c-8ba5-b99bdcb8e158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:34.499321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d7d41907-ceab-4c3c-8ba5-b99bdcb8e158 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:34.500519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d7d41907-ceab-4c3c-8ba5-b99bdcb8e158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:34.509707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 711/1399] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:34.566015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa9ed-e1e3-4b85-a4f6-514292526c80 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:34.566301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa9ed-e1e3-4b85-a4f6-514292526c80 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:34.566766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa9ed-e1e3-4b85-a4f6-514292526c80 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:34.568751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 689/1400] 10.222.0.33 () {54 vars in 962 bytes} [Thu Jun 23 08:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:34.573919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb85d0-f1c5-49ce-98ce-d2c38435175a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:34.574208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb85d0-f1c5-49ce-98ce-d2c38435175a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:34.574482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb85d0-f1c5-49ce-98ce-d2c38435175a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:34.625008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-83eb85d0-f1c5-49ce-98ce-d2c38435175a None None] MFA Rules not processed for user `d8cbcc9295074710aeacb35e4e1a4348`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:34.626206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-83eb85d0-f1c5-49ce-98ce-d2c38435175a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:34.662884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:13:34.693778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 712/1401] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Jun 23 08:13:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:34.701435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaa9ed-e1e3-4b85-a4f6-514292526c80 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:34.714501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaa9ed-e1e3-4b85-a4f6-514292526c80 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:34.718868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d04c8-70bb-440d-b8e1-5d8108347075 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8cbcc9295074710aeacb35e4e1a4348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:34.719566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d04c8-70bb-440d-b8e1-5d8108347075 service glance] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:34.719847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d04c8-70bb-440d-b8e1-5d8108347075 service glance] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:34.720122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d04c8-70bb-440d-b8e1-5d8108347075 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:34.730803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d04c8-70bb-440d-b8e1-5d8108347075 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:34.732227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d04c8-70bb-440d-b8e1-5d8108347075 service glance] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:34.742844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 690/1402] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:35.451256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef358e75-0075-43d3-aef2-40128560f52f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:35.451550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef358e75-0075-43d3-aef2-40128560f52f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:35.451849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef358e75-0075-43d3-aef2-40128560f52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:35.502791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ef358e75-0075-43d3-aef2-40128560f52f None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:35.504203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef358e75-0075-43d3-aef2-40128560f52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:35.568356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd00b-2d80-45e5-9f00-0ded5fa3957d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:35.568910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd00b-2d80-45e5-9f00-0ded5fa3957d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:35.568910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd00b-2d80-45e5-9f00-0ded5fa3957d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:35.570288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 691/1403] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:13:35.574079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 713/1404] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:13:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:35.575714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac383-7083-4d71-990d-50a89a14dc3f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:35.576000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac383-7083-4d71-990d-50a89a14dc3f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:35.576406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac383-7083-4d71-990d-50a89a14dc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:35.595115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ddcac383-7083-4d71-990d-50a89a14dc3f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:35.596438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ddcac383-7083-4d71-990d-50a89a14dc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:35.604638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 692/1405] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:13:35.619250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963a77d-e0cc-43f0-b483-21dedb1122d4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:35.619583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963a77d-e0cc-43f0-b483-21dedb1122d4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:35.619862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963a77d-e0cc-43f0-b483-21dedb1122d4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:35.621323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 714/1406] 10.222.0.33 () {54 vars in 942 bytes} [Thu Jun 23 08:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:35.627829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862a98-711e-4811-b415-860910adcbc2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:35.628062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862a98-711e-4811-b415-860910adcbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:35.628338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862a98-711e-4811-b415-860910adcbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:35.650696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963a77d-e0cc-43f0-b483-21dedb1122d4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:35.665958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963a77d-e0cc-43f0-b483-21dedb1122d4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:35.671746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc4be6-7239-4ed3-994a-43d942f04f7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:35.672391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc4be6-7239-4ed3-994a-43d942f04f7d service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:35.672676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc4be6-7239-4ed3-994a-43d942f04f7d service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:35.672955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc4be6-7239-4ed3-994a-43d942f04f7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:35.677551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fd862a98-711e-4811-b415-860910adcbc2 None None] MFA Rules not processed for user `7356a0f8d9f84e7cb785662db9afd658`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:35.678790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fd862a98-711e-4811-b415-860910adcbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:35.681626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc4be6-7239-4ed3-994a-43d942f04f7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:35.683240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc4be6-7239-4ed3-994a-43d942f04f7d service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:35.692720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 715/1407] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:13:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:13:35.732071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:13:35.756542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 693/1408] 10.222.0.33 () {58 vars in 1039 bytes} [Thu Jun 23 08:13:35 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:35.766357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc4be6-7239-4ed3-994a-43d942f04f7d service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:13:35.778206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc4be6-7239-4ed3-994a-43d942f04f7d service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:13:35.783883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20a4fa-3332-45b5-a997-c0f9efafb690 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:13:35.784557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20a4fa-3332-45b5-a997-c0f9efafb690 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:35.784871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20a4fa-3332-45b5-a997-c0f9efafb690 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:35.785122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20a4fa-3332-45b5-a997-c0f9efafb690 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:35.794859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20a4fa-3332-45b5-a997-c0f9efafb690 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:13:35.796683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20a4fa-3332-45b5-a997-c0f9efafb690 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:13:35.806024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 716/1409] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:13:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:13:45.753648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:13:45.802145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=62 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:13:57.694426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3574f-ddde-4411-a49f-87fa4b90b7c6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:57.695070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3574f-ddde-4411-a49f-87fa4b90b7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:57.695070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3574f-ddde-4411-a49f-87fa4b90b7c6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:57.696531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 694/1410] 10.222.0.33 () {54 vars in 918 bytes} [Thu Jun 23 08:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:57.701656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5872-254e-4038-b6ce-75ac1917ff44 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:57.702122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5872-254e-4038-b6ce-75ac1917ff44 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:57.702225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5872-254e-4038-b6ce-75ac1917ff44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:57.721041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fd4c5872-254e-4038-b6ce-75ac1917ff44 None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:57.722354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fd4c5872-254e-4038-b6ce-75ac1917ff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:57.731737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 717/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 23 08:13:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:13:57.747234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f224a21-1b95-433c-acf9-7b1c701dd279 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:57.747546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f224a21-1b95-433c-acf9-7b1c701dd279 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:57.747822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f224a21-1b95-433c-acf9-7b1c701dd279 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:57.749244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 695/1412] 10.222.0.33 () {54 vars in 918 bytes} [Thu Jun 23 08:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:13:57.754241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c5dd0-2b24-4f4a-81ea-46d82b60bd8f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:13:57.754523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c5dd0-2b24-4f4a-81ea-46d82b60bd8f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:13:57.754804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c5dd0-2b24-4f4a-81ea-46d82b60bd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:13:57.774153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e83c5dd0-2b24-4f4a-81ea-46d82b60bd8f None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:13:57.776110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e83c5dd0-2b24-4f4a-81ea-46d82b60bd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:13:57.786339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 718/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 23 08:13:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:14:00.256323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d42a7-336b-4390-bd96-344fec05d853 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:00.256940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d42a7-336b-4390-bd96-344fec05d853 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:00.256940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d42a7-336b-4390-bd96-344fec05d853 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:00.258247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:00.264783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee1e98-0006-4a8c-915a-07fa0aceb933 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:00.265241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee1e98-0006-4a8c-915a-07fa0aceb933 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:00.265348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee1e98-0006-4a8c-915a-07fa0aceb933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:00.291072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-feee1e98-0006-4a8c-915a-07fa0aceb933 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:00.292297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-feee1e98-0006-4a8c-915a-07fa0aceb933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:00.301658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:00.346131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d42a7-336b-4390-bd96-344fec05d853 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:00.362172 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d42a7-336b-4390-bd96-344fec05d853 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:00.368275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a66e2-937c-4164-a8c1-cd5f96a52f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:00.368918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a66e2-937c-4164-a8c1-cd5f96a52f84 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:00.369201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a66e2-937c-4164-a8c1-cd5f96a52f84 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:00.369474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a66e2-937c-4164-a8c1-cd5f96a52f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:00.377675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a66e2-937c-4164-a8c1-cd5f96a52f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:00.379377 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a66e2-937c-4164-a8c1-cd5f96a52f84 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:00.388627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 697/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:01.285348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a003-7bba-4bfb-ab8f-6ca1099eab5d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:01.285941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a003-7bba-4bfb-ab8f-6ca1099eab5d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:01.285941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a003-7bba-4bfb-ab8f-6ca1099eab5d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:01.287292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:01.294384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52521a-1e28-4415-bdb2-17a6b2a2c393 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:01.294848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52521a-1e28-4415-bdb2-17a6b2a2c393 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:01.294943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52521a-1e28-4415-bdb2-17a6b2a2c393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:01.314756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4f52521a-1e28-4415-bdb2-17a6b2a2c393 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:01.315901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4f52521a-1e28-4415-bdb2-17a6b2a2c393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:01.324199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:01.472974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646a003-7bba-4bfb-ab8f-6ca1099eab5d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:01.493213 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646a003-7bba-4bfb-ab8f-6ca1099eab5d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:01.498846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d99dda-de0a-4fce-bb2d-186f456fd6b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:01.499524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d99dda-de0a-4fce-bb2d-186f456fd6b3 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:01.499802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d99dda-de0a-4fce-bb2d-186f456fd6b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:01.500070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d99dda-de0a-4fce-bb2d-186f456fd6b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:01.509037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d99dda-de0a-4fce-bb2d-186f456fd6b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:01.510559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d99dda-de0a-4fce-bb2d-186f456fd6b3 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:01.520870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 721/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:02.407589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547608e-d610-4648-892d-1b94be62c475 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:02.408204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547608e-d610-4648-892d-1b94be62c475 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:02.408204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547608e-d610-4648-892d-1b94be62c475 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:02.409498 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:02.415444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cec37-e5d8-4163-873b-a121a708ef44 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:02.415898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cec37-e5d8-4163-873b-a121a708ef44 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:02.416005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cec37-e5d8-4163-873b-a121a708ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:02.438714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cd1cec37-e5d8-4163-873b-a121a708ef44 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:02.440580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cd1cec37-e5d8-4163-873b-a121a708ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:02.449990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:02.601340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547608e-d610-4648-892d-1b94be62c475 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:02.626921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547608e-d610-4648-892d-1b94be62c475 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:02.634507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39290c84-c47e-4785-811c-559cfcbd2028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:02.635151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290c84-c47e-4785-811c-559cfcbd2028 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:02.635466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290c84-c47e-4785-811c-559cfcbd2028 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:02.635740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290c84-c47e-4785-811c-559cfcbd2028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:02.644683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39290c84-c47e-4785-811c-559cfcbd2028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:02.646302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39290c84-c47e-4785-811c-559cfcbd2028 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:02.656440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 700/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:03.517206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d52577-a8cc-420d-805c-2e01b1c53093 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:03.517736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d52577-a8cc-420d-805c-2e01b1c53093 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:03.517736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d52577-a8cc-420d-805c-2e01b1c53093 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:03.519247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:03.524682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565707b-7347-46ae-96d4-fd2843c884d9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:03.525104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565707b-7347-46ae-96d4-fd2843c884d9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:03.525241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565707b-7347-46ae-96d4-fd2843c884d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:03.544089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8565707b-7347-46ae-96d4-fd2843c884d9 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:03.545344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8565707b-7347-46ae-96d4-fd2843c884d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:03.554626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:03.703473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d52577-a8cc-420d-805c-2e01b1c53093 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:03.720384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d52577-a8cc-420d-805c-2e01b1c53093 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:03.729012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429318e-840a-4663-bb40-06ccca051442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:03.729946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429318e-840a-4663-bb40-06ccca051442 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:03.730308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429318e-840a-4663-bb40-06ccca051442 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:03.730680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429318e-840a-4663-bb40-06ccca051442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:03.743108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429318e-840a-4663-bb40-06ccca051442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:03.744761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429318e-840a-4663-bb40-06ccca051442 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:03.754919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 724/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:04.614384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae082c3-bcd5-4f48-bad5-18daf3a72c15 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:04.614948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae082c3-bcd5-4f48-bad5-18daf3a72c15 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:04.614948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae082c3-bcd5-4f48-bad5-18daf3a72c15 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:04.616901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:04.623012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37331376-4833-475a-b318-4b093346fb9f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:04.623509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37331376-4833-475a-b318-4b093346fb9f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:04.623608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37331376-4833-475a-b318-4b093346fb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:04.650214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-37331376-4833-475a-b318-4b093346fb9f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:04.651437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-37331376-4833-475a-b318-4b093346fb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:04.659730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:04.819052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae082c3-bcd5-4f48-bad5-18daf3a72c15 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:04.844460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae082c3-bcd5-4f48-bad5-18daf3a72c15 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:04.852404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837277ae-c6a6-49f6-8497-71b1fb601a69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:04.853052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837277ae-c6a6-49f6-8497-71b1fb601a69 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:04.853336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837277ae-c6a6-49f6-8497-71b1fb601a69 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:04.853644 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837277ae-c6a6-49f6-8497-71b1fb601a69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:04.862649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837277ae-c6a6-49f6-8497-71b1fb601a69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:04.864869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837277ae-c6a6-49f6-8497-71b1fb601a69 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:04.874798 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 703/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:05.682650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964af3a-d8a6-403f-b89e-e304bf0fcf39 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:05.683273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964af3a-d8a6-403f-b89e-e304bf0fcf39 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:05.683273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964af3a-d8a6-403f-b89e-e304bf0fcf39 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:05.684567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:05.690749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724b682-e238-4b47-8e8c-cf6401656dbf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:05.691651 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724b682-e238-4b47-8e8c-cf6401656dbf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:05.692133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724b682-e238-4b47-8e8c-cf6401656dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:05.721467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c724b682-e238-4b47-8e8c-cf6401656dbf None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:05.722978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c724b682-e238-4b47-8e8c-cf6401656dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:05.732431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:05.887020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964af3a-d8a6-403f-b89e-e304bf0fcf39 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:05.906821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964af3a-d8a6-403f-b89e-e304bf0fcf39 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:05.913920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc2228-aa46-4bc1-9e0e-61a2f0e9cef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:05.914559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc2228-aa46-4bc1-9e0e-61a2f0e9cef2 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:05.914863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc2228-aa46-4bc1-9e0e-61a2f0e9cef2 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:05.915117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc2228-aa46-4bc1-9e0e-61a2f0e9cef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:05.924567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc2228-aa46-4bc1-9e0e-61a2f0e9cef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:05.926411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc2228-aa46-4bc1-9e0e-61a2f0e9cef2 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:05.935966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 727/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:14:06.737529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3342e-eaba-4e74-a3ce-800377d6b02b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:06.738444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3342e-eaba-4e74-a3ce-800377d6b02b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:06.738444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3342e-eaba-4e74-a3ce-800377d6b02b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:06.740433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:14:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:06.748411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645ad23-ca3a-48ee-853f-33f39fd56033 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:06.748963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645ad23-ca3a-48ee-853f-33f39fd56033 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:06.748963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645ad23-ca3a-48ee-853f-33f39fd56033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:06.775759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4645ad23-ca3a-48ee-853f-33f39fd56033 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:06.776896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4645ad23-ca3a-48ee-853f-33f39fd56033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:06.786671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:06.942181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3342e-eaba-4e74-a3ce-800377d6b02b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:06.957440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3342e-eaba-4e74-a3ce-800377d6b02b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:06.963747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d591-c70d-49b7-8edc-5de2263fc4ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:06.964380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d591-c70d-49b7-8edc-5de2263fc4ec service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:06.964666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d591-c70d-49b7-8edc-5de2263fc4ec service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:06.964934 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d591-c70d-49b7-8edc-5de2263fc4ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:06.973435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d591-c70d-49b7-8edc-5de2263fc4ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:06.975005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d591-c70d-49b7-8edc-5de2263fc4ec service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:06.984198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 706/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:07.780044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbe91-38eb-4200-9073-aa4fc09fbd6a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:07.780942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbe91-38eb-4200-9073-aa4fc09fbd6a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:07.780942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbe91-38eb-4200-9073-aa4fc09fbd6a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:07.783044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:14:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:07.788326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13070bf0-ac8c-47f3-b733-ef264d32f72e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:07.788834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13070bf0-ac8c-47f3-b733-ef264d32f72e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:07.788834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13070bf0-ac8c-47f3-b733-ef264d32f72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:07.808253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-13070bf0-ac8c-47f3-b733-ef264d32f72e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:07.809456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-13070bf0-ac8c-47f3-b733-ef264d32f72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:07.818956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:07.972918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbe91-38eb-4200-9073-aa4fc09fbd6a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:07.989834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbe91-38eb-4200-9073-aa4fc09fbd6a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:07.996541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f819b7-b4e5-4984-8b2a-33239c6d9960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:07.997174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f819b7-b4e5-4984-8b2a-33239c6d9960 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:07.997417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f819b7-b4e5-4984-8b2a-33239c6d9960 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:07.997707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f819b7-b4e5-4984-8b2a-33239c6d9960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:08.006808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f819b7-b4e5-4984-8b2a-33239c6d9960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:08.008426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f819b7-b4e5-4984-8b2a-33239c6d9960 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:08.018315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 730/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:08.826183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd2a0-0f22-4eb5-989e-26aaf3b8dab8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:08.826752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd2a0-0f22-4eb5-989e-26aaf3b8dab8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:08.826752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd2a0-0f22-4eb5-989e-26aaf3b8dab8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:08.828033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:08.834186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1eec7-a8e7-4fa7-a585-5c59a27a8a26 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:08.834693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1eec7-a8e7-4fa7-a585-5c59a27a8a26 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:08.834802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1eec7-a8e7-4fa7-a585-5c59a27a8a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:08.861802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32b1eec7-a8e7-4fa7-a585-5c59a27a8a26 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:08.862981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32b1eec7-a8e7-4fa7-a585-5c59a27a8a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:08.871774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:09.021888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfd2a0-0f22-4eb5-989e-26aaf3b8dab8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:09.048131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfd2a0-0f22-4eb5-989e-26aaf3b8dab8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:09.056087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8a21d-728e-4836-af1f-925179035668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:09.056727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8a21d-728e-4836-af1f-925179035668 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:09.057010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8a21d-728e-4836-af1f-925179035668 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:09.057287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8a21d-728e-4836-af1f-925179035668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:09.065821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8a21d-728e-4836-af1f-925179035668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:09.067547 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8a21d-728e-4836-af1f-925179035668 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:09.077582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 709/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:11.008220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2eab3-260e-4a27-adce-f2e7e9907024 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:11.009143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2eab3-260e-4a27-adce-f2e7e9907024 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:11.009143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2eab3-260e-4a27-adce-f2e7e9907024 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:11.011269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:11.019368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8fda-ed88-4559-8043-285a63dbe4cf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:11.019893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8fda-ed88-4559-8043-285a63dbe4cf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:11.020067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8fda-ed88-4559-8043-285a63dbe4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:11.039689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-170d8fda-ed88-4559-8043-285a63dbe4cf None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:11.040867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-170d8fda-ed88-4559-8043-285a63dbe4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:11.049583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:14:11.092157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2eab3-260e-4a27-adce-f2e7e9907024 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:11.104145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2eab3-260e-4a27-adce-f2e7e9907024 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:11.108030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6f512-d5de-48b9-927b-a3fc537752c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:11.108664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6f512-d5de-48b9-927b-a3fc537752c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:11.108919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6f512-d5de-48b9-927b-a3fc537752c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:11.109185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6f512-d5de-48b9-927b-a3fc537752c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:11.120121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6f512-d5de-48b9-927b-a3fc537752c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:11.121566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6f512-d5de-48b9-927b-a3fc537752c7 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:11.131473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 733/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:13.308470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f8215-8d7d-49e9-ba76-07c142dcd7be None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:13.309078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f8215-8d7d-49e9-ba76-07c142dcd7be None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:13.309078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f8215-8d7d-49e9-ba76-07c142dcd7be None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:13.310383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:13.317064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef5fd3-e940-4880-a065-c5b21d2b13ea None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:13.317889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef5fd3-e940-4880-a065-c5b21d2b13ea None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:13.318153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef5fd3-e940-4880-a065-c5b21d2b13ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:13.339198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e3ef5fd3-e940-4880-a065-c5b21d2b13ea None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:13.340407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e3ef5fd3-e940-4880-a065-c5b21d2b13ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:13.349173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:13.390935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f8215-8d7d-49e9-ba76-07c142dcd7be None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:13.405295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f8215-8d7d-49e9-ba76-07c142dcd7be None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:13.409785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa80d3-eb1b-460b-87ef-32160ca72b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:13.410430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa80d3-eb1b-460b-87ef-32160ca72b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:13.410713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa80d3-eb1b-460b-87ef-32160ca72b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:13.411102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa80d3-eb1b-460b-87ef-32160ca72b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:13.420999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa80d3-eb1b-460b-87ef-32160ca72b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:13.422447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa80d3-eb1b-460b-87ef-32160ca72b1f service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:13.432390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 712/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:15.810246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99d0e1-cb3e-4d92-b844-b2f1004e4063 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:15.810810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99d0e1-cb3e-4d92-b844-b2f1004e4063 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:15.810810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99d0e1-cb3e-4d92-b844-b2f1004e4063 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:15.812221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:15.818054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b225-3041-4802-8389-86836709dcff None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:15.818511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b225-3041-4802-8389-86836709dcff None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:15.818607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b225-3041-4802-8389-86836709dcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:15.836860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3ce7b225-3041-4802-8389-86836709dcff None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:15.838280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3ce7b225-3041-4802-8389-86836709dcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:15.847182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:15.892317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99d0e1-cb3e-4d92-b844-b2f1004e4063 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:15.905868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99d0e1-cb3e-4d92-b844-b2f1004e4063 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:15.910167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6c8ed-6212-4d2f-84ed-15e96d8e7e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:15.910865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c8ed-6212-4d2f-84ed-15e96d8e7e8f service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:15.911249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c8ed-6212-4d2f-84ed-15e96d8e7e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:15.911535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6c8ed-6212-4d2f-84ed-15e96d8e7e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:15.920723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6c8ed-6212-4d2f-84ed-15e96d8e7e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:15.922171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6c8ed-6212-4d2f-84ed-15e96d8e7e8f service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:15.931731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 736/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:18.066490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c78c5-52bf-407c-8514-d3063c147fb4 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:18.067139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c78c5-52bf-407c-8514-d3063c147fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:18.067139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c78c5-52bf-407c-8514-d3063c147fb4 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:18.068399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:18.074892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289c749-c106-4dd4-b255-58f53aceea0d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:18.075603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289c749-c106-4dd4-b255-58f53aceea0d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:18.075790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289c749-c106-4dd4-b255-58f53aceea0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:18.097465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6289c749-c106-4dd4-b255-58f53aceea0d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:18.098781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6289c749-c106-4dd4-b255-58f53aceea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:18.108247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:18.147500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c78c5-52bf-407c-8514-d3063c147fb4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:18.163566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c78c5-52bf-407c-8514-d3063c147fb4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:18.167969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce1018-dc16-4e0e-947c-85a50cd5acbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:18.168602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1018-dc16-4e0e-947c-85a50cd5acbd service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:18.168908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1018-dc16-4e0e-947c-85a50cd5acbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:18.169162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1018-dc16-4e0e-947c-85a50cd5acbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:18.182286 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce1018-dc16-4e0e-947c-85a50cd5acbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:18.184509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce1018-dc16-4e0e-947c-85a50cd5acbd service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:18.197169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 715/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:20.298917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e76bc0-4473-4e91-a02c-225af26ad1b0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:20.299840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e76bc0-4473-4e91-a02c-225af26ad1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:20.299840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e76bc0-4473-4e91-a02c-225af26ad1b0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:20.301892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:20.309569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccf325-37f7-4cc8-ba26-ad234f11e727 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:20.310024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccf325-37f7-4cc8-ba26-ad234f11e727 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:20.310142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccf325-37f7-4cc8-ba26-ad234f11e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:20.330427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e5ccf325-37f7-4cc8-ba26-ad234f11e727 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:20.331671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e5ccf325-37f7-4cc8-ba26-ad234f11e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:20.340242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:20.381871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e76bc0-4473-4e91-a02c-225af26ad1b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:20.399435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e76bc0-4473-4e91-a02c-225af26ad1b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:20.403914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15034d9-6779-4192-a2e6-8cf4089f4673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:20.404561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15034d9-6779-4192-a2e6-8cf4089f4673 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:20.404840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15034d9-6779-4192-a2e6-8cf4089f4673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:20.405112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15034d9-6779-4192-a2e6-8cf4089f4673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:20.414792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15034d9-6779-4192-a2e6-8cf4089f4673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:20.416281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15034d9-6779-4192-a2e6-8cf4089f4673 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:20.426157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 739/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:22.501628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf8f1-dedb-49f6-b64f-1ca6729a6925 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:22.502240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf8f1-dedb-49f6-b64f-1ca6729a6925 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:22.502240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf8f1-dedb-49f6-b64f-1ca6729a6925 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:22.503778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:22.509809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de2d5-7848-44b4-85b2-c5c6591deb3e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:22.510228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de2d5-7848-44b4-85b2-c5c6591deb3e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:22.510357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de2d5-7848-44b4-85b2-c5c6591deb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:22.529671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6c5de2d5-7848-44b4-85b2-c5c6591deb3e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:22.530864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6c5de2d5-7848-44b4-85b2-c5c6591deb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:22.539593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:22.585100 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bf8f1-dedb-49f6-b64f-1ca6729a6925 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:22.599163 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bf8f1-dedb-49f6-b64f-1ca6729a6925 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:22.606085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc822e4e-580e-4420-b41f-7ba132defe70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:22.606928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822e4e-580e-4420-b41f-7ba132defe70 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:22.607339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822e4e-580e-4420-b41f-7ba132defe70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:22.607607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822e4e-580e-4420-b41f-7ba132defe70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:22.620226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc822e4e-580e-4420-b41f-7ba132defe70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:22.621662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc822e4e-580e-4420-b41f-7ba132defe70 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:22.631642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 718/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:24.740922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e38a42-6098-40c4-b4e9-7aca84485088 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:24.741524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e38a42-6098-40c4-b4e9-7aca84485088 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:24.741524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e38a42-6098-40c4-b4e9-7aca84485088 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:24.743141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:24.749492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67571f-66c2-41df-be15-835000dc49ec None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:24.749961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67571f-66c2-41df-be15-835000dc49ec None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:24.750071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67571f-66c2-41df-be15-835000dc49ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:24.776574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2d67571f-66c2-41df-be15-835000dc49ec None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:24.777701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2d67571f-66c2-41df-be15-835000dc49ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:24.786975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:24.827897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e38a42-6098-40c4-b4e9-7aca84485088 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:24.846381 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e38a42-6098-40c4-b4e9-7aca84485088 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:24.853021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba44f5-2637-4f06-81e1-3387d26833d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:24.853981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba44f5-2637-4f06-81e1-3387d26833d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:24.854378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba44f5-2637-4f06-81e1-3387d26833d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:24.854779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba44f5-2637-4f06-81e1-3387d26833d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:24.868839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba44f5-2637-4f06-81e1-3387d26833d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:24.870540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba44f5-2637-4f06-81e1-3387d26833d7 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:24.880121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 742/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:26.908508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a25ad1-a98d-43f9-b62e-5d2c6ac83d49 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:26.909107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a25ad1-a98d-43f9-b62e-5d2c6ac83d49 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:26.909107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a25ad1-a98d-43f9-b62e-5d2c6ac83d49 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:26.910438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:26.916833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d78883-0712-4c3c-8ac0-56fe234e4449 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:26.917733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d78883-0712-4c3c-8ac0-56fe234e4449 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:26.917733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d78883-0712-4c3c-8ac0-56fe234e4449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:26.928147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=79 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:14:26.941578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-80d78883-0712-4c3c-8ac0-56fe234e4449 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:26.942790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80d78883-0712-4c3c-8ac0-56fe234e4449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:26.950578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:26.988012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a25ad1-a98d-43f9-b62e-5d2c6ac83d49 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:26.994308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=73 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:14:27.001093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a25ad1-a98d-43f9-b62e-5d2c6ac83d49 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:27.004774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ace399-de05-4687-86cd-717b8408b51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:27.005773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ace399-de05-4687-86cd-717b8408b51b service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:27.006057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ace399-de05-4687-86cd-717b8408b51b service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:27.006352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ace399-de05-4687-86cd-717b8408b51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:27.015522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ace399-de05-4687-86cd-717b8408b51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:27.016946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ace399-de05-4687-86cd-717b8408b51b service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:27.026375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 721/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:29.061848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c758d9-d649-44e4-840e-ccb9c4593985 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:29.062757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c758d9-d649-44e4-840e-ccb9c4593985 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:29.062757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c758d9-d649-44e4-840e-ccb9c4593985 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:29.064302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:29.069841 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd795-c719-49be-8a73-a00663c941da None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:29.070305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd795-c719-49be-8a73-a00663c941da None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:29.070404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd795-c719-49be-8a73-a00663c941da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:29.091857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d31cd795-c719-49be-8a73-a00663c941da None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:29.093114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d31cd795-c719-49be-8a73-a00663c941da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:29.101874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:14:29.139520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c758d9-d649-44e4-840e-ccb9c4593985 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:29.153609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c758d9-d649-44e4-840e-ccb9c4593985 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:29.157790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee2eff-227e-455d-b614-a82adf839835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:29.158432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee2eff-227e-455d-b614-a82adf839835 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:29.158713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee2eff-227e-455d-b614-a82adf839835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:29.158984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee2eff-227e-455d-b614-a82adf839835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:29.169309 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee2eff-227e-455d-b614-a82adf839835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:29.170757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee2eff-227e-455d-b614-a82adf839835 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:29.182140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 745/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:31.290279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d06bd-3bdd-42eb-a972-c23a216a73d6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:31.290843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d06bd-3bdd-42eb-a972-c23a216a73d6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:31.290843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d06bd-3bdd-42eb-a972-c23a216a73d6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:31.292264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:31.304094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1af13-f370-4ad8-9e81-b08356c2f225 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:31.304548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1af13-f370-4ad8-9e81-b08356c2f225 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:31.304659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1af13-f370-4ad8-9e81-b08356c2f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:31.333103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7ad1af13-f370-4ad8-9e81-b08356c2f225 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:31.334304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ad1af13-f370-4ad8-9e81-b08356c2f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:31.343854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:31.389542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d06bd-3bdd-42eb-a972-c23a216a73d6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:31.403653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d06bd-3bdd-42eb-a972-c23a216a73d6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:31.407669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde31f01-170d-40c8-a45c-a2344ec8bfe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:31.408311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31f01-170d-40c8-a45c-a2344ec8bfe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:31.408595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31f01-170d-40c8-a45c-a2344ec8bfe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:31.408867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31f01-170d-40c8-a45c-a2344ec8bfe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:31.418840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde31f01-170d-40c8-a45c-a2344ec8bfe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:31.420322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde31f01-170d-40c8-a45c-a2344ec8bfe4 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:31.430084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 724/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:41.324282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=11 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:14:41.425655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=14 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:14:51.144402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a555dc-1d60-4333-8c01-9bc9f782b22d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:51.145015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a555dc-1d60-4333-8c01-9bc9f782b22d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:51.145015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a555dc-1d60-4333-8c01-9bc9f782b22d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:51.147113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 747/1471] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:14:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:51.153073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b6d3-ccac-48b1-8362-119523edde68 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:51.153882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b6d3-ccac-48b1-8362-119523edde68 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:51.154325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b6d3-ccac-48b1-8362-119523edde68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:51.173890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9119b6d3-ccac-48b1-8362-119523edde68 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:51.175099 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9119b6d3-ccac-48b1-8362-119523edde68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:51.183867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 725/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:14:51.232813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38837341-b77c-42fd-968d-56e561fd9a9f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:51.233135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38837341-b77c-42fd-968d-56e561fd9a9f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:51.233457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38837341-b77c-42fd-968d-56e561fd9a9f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:51.235412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 748/1473] 10.222.0.33 () {54 vars in 962 bytes} [Thu Jun 23 08:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:51.240400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c67657-00af-43e8-85dc-83e3f8a45c82 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:51.240688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c67657-00af-43e8-85dc-83e3f8a45c82 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:51.240967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c67657-00af-43e8-85dc-83e3f8a45c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:51.265361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-06c67657-00af-43e8-85dc-83e3f8a45c82 None None] MFA Rules not processed for user `d8cbcc9295074710aeacb35e4e1a4348`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:51.267071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-06c67657-00af-43e8-85dc-83e3f8a45c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:51.276280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 726/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Jun 23 08:14:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:51.283761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38837341-b77c-42fd-968d-56e561fd9a9f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:51.295236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38837341-b77c-42fd-968d-56e561fd9a9f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:51.298842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc6cda-ac49-4f8d-88d2-043dd00c7b42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8cbcc9295074710aeacb35e4e1a4348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:51.299510 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc6cda-ac49-4f8d-88d2-043dd00c7b42 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:51.299799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc6cda-ac49-4f8d-88d2-043dd00c7b42 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:51.300068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc6cda-ac49-4f8d-88d2-043dd00c7b42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:51.309004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc6cda-ac49-4f8d-88d2-043dd00c7b42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:51.310444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc6cda-ac49-4f8d-88d2-043dd00c7b42 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:51.319947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 749/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:14:53.460349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb0a09-010d-4164-9146-84ffe02cbf50 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:53.461310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb0a09-010d-4164-9146-84ffe02cbf50 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:53.461310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb0a09-010d-4164-9146-84ffe02cbf50 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:53.463542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 727/1476] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:14:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:53.469790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df0ac-dffe-41f4-99db-6c02f4fee619 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:53.470502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df0ac-dffe-41f4-99db-6c02f4fee619 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:53.470680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df0ac-dffe-41f4-99db-6c02f4fee619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:53.498398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-947df0ac-dffe-41f4-99db-6c02f4fee619 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:53.499643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-947df0ac-dffe-41f4-99db-6c02f4fee619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:53.509379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 750/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:53.533301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb0a09-010d-4164-9146-84ffe02cbf50 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:53.547610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb0a09-010d-4164-9146-84ffe02cbf50 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:53.553548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0de98d-0e24-4183-a92e-cd74451d7672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:53.554210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de98d-0e24-4183-a92e-cd74451d7672 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:53.554491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de98d-0e24-4183-a92e-cd74451d7672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:53.554762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de98d-0e24-4183-a92e-cd74451d7672 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:53.555538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0de98d-0e24-4183-a92e-cd74451d7672 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:53.557331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-5a0de98d-0e24-4183-a92e-cd74451d7672 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:14:53.557331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jun 23 08:14:53.557331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: Jun 23 08:14:53.558533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0de98d-0e24-4183-a92e-cd74451d7672 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:53.578494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 728/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 08:14:53 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:14:55.617636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03b89c-fcb4-48a4-846b-816d93590b58 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:55.618296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03b89c-fcb4-48a4-846b-816d93590b58 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:55.618296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03b89c-fcb4-48a4-846b-816d93590b58 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:55.619607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 751/1479] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:55.626038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c305f7-90d3-4e4c-9b69-299c952173bb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:55.626833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c305f7-90d3-4e4c-9b69-299c952173bb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:55.627005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c305f7-90d3-4e4c-9b69-299c952173bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:55.648768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-45c305f7-90d3-4e4c-9b69-299c952173bb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:55.650070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-45c305f7-90d3-4e4c-9b69-299c952173bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:55.659992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 729/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:55.708686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03b89c-fcb4-48a4-846b-816d93590b58 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:55.723747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03b89c-fcb4-48a4-846b-816d93590b58 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:55.729863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3179b7-120c-4f50-a71f-013a111eaddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:55.730502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3179b7-120c-4f50-a71f-013a111eaddd service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:55.730779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3179b7-120c-4f50-a71f-013a111eaddd service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:55.731049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3179b7-120c-4f50-a71f-013a111eaddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:55.739368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3179b7-120c-4f50-a71f-013a111eaddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:55.741024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3179b7-120c-4f50-a71f-013a111eaddd service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:55.751586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 752/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:14:57.945782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9def8b-a9cc-4959-a9fc-53d89ba5b4ac None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:57.946708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9def8b-a9cc-4959-a9fc-53d89ba5b4ac None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:57.946708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9def8b-a9cc-4959-a9fc-53d89ba5b4ac None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:57.948872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 730/1482] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:14:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:14:57.956759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801563a-57d4-4c56-8cf2-fd24dee19164 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:57.957243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801563a-57d4-4c56-8cf2-fd24dee19164 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:57.957373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801563a-57d4-4c56-8cf2-fd24dee19164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:57.976329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9801563a-57d4-4c56-8cf2-fd24dee19164 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:14:57.977524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9801563a-57d4-4c56-8cf2-fd24dee19164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:14:57.986129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 753/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:14:58.023124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9def8b-a9cc-4959-a9fc-53d89ba5b4ac None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:14:58.035084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9def8b-a9cc-4959-a9fc-53d89ba5b4ac None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:14:58.039581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120673b0-0949-44ae-ac3b-3edf9c78b648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:14:58.040280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120673b0-0949-44ae-ac3b-3edf9c78b648 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:14:58.040542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120673b0-0949-44ae-ac3b-3edf9c78b648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:14:58.040814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120673b0-0949-44ae-ac3b-3edf9c78b648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:14:58.053942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120673b0-0949-44ae-ac3b-3edf9c78b648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:14:58.056312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120673b0-0949-44ae-ac3b-3edf9c78b648 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:14:58.071732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 731/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:15:00.145425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66ba0f-1cd5-4e96-88ad-d75e3d1b0306 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:00.146333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66ba0f-1cd5-4e96-88ad-d75e3d1b0306 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:00.146333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66ba0f-1cd5-4e96-88ad-d75e3d1b0306 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:00.149553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 754/1485] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:15:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:15:00.156120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1775c4-b080-420d-a30d-16a90b3c5733 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:00.156584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1775c4-b080-420d-a30d-16a90b3c5733 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:00.156699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1775c4-b080-420d-a30d-16a90b3c5733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:00.177843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6e1775c4-b080-420d-a30d-16a90b3c5733 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:15:00.179440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6e1775c4-b080-420d-a30d-16a90b3c5733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:15:00.188261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 732/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:15:00.237099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66ba0f-1cd5-4e96-88ad-d75e3d1b0306 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:15:00.251401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66ba0f-1cd5-4e96-88ad-d75e3d1b0306 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:15:00.255393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44623e3-0fd4-4067-bc45-bf5e97773e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:15:00.256503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44623e3-0fd4-4067-bc45-bf5e97773e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:00.256801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44623e3-0fd4-4067-bc45-bf5e97773e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:00.257082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44623e3-0fd4-4067-bc45-bf5e97773e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:00.267102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44623e3-0fd4-4067-bc45-bf5e97773e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:15:00.268602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44623e3-0fd4-4067-bc45-bf5e97773e84 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:15:00.279340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 755/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:15:02.461203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dafa26-e1c8-44e2-9e56-83efb40e618c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:02.461793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dafa26-e1c8-44e2-9e56-83efb40e618c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:02.461793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dafa26-e1c8-44e2-9e56-83efb40e618c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:02.464013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 733/1488] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:15:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:15:02.471972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682249f-4605-449e-afc8-74bfca1060d6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:02.472442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682249f-4605-449e-afc8-74bfca1060d6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:02.472554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682249f-4605-449e-afc8-74bfca1060d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:02.490953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1682249f-4605-449e-afc8-74bfca1060d6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:15:02.492525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1682249f-4605-449e-afc8-74bfca1060d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:15:02.501102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 756/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:15:02.543573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dafa26-e1c8-44e2-9e56-83efb40e618c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:15:02.555894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dafa26-e1c8-44e2-9e56-83efb40e618c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:15:02.559923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239646d3-2a51-4d03-9374-2b3dfae57e69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:15:02.560554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239646d3-2a51-4d03-9374-2b3dfae57e69 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:02.560830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239646d3-2a51-4d03-9374-2b3dfae57e69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:02.561098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239646d3-2a51-4d03-9374-2b3dfae57e69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:02.573205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239646d3-2a51-4d03-9374-2b3dfae57e69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:15:02.575527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239646d3-2a51-4d03-9374-2b3dfae57e69 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:15:02.588969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 734/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:15:04.800219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e55d-dad5-4e23-ad3b-ea806a21adc1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:04.801147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e55d-dad5-4e23-ad3b-ea806a21adc1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:04.801147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e55d-dad5-4e23-ad3b-ea806a21adc1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:04.803888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 757/1491] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 08:15:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:15:04.809295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c49bb-9c75-47ce-ad3c-ce3eeead5bf9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:04.809759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c49bb-9c75-47ce-ad3c-ce3eeead5bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:04.809869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c49bb-9c75-47ce-ad3c-ce3eeead5bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:04.829359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1b2c49bb-9c75-47ce-ad3c-ce3eeead5bf9 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:15:04.830561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1b2c49bb-9c75-47ce-ad3c-ce3eeead5bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:15:04.839376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 735/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 08:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:15:04.881654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e55d-dad5-4e23-ad3b-ea806a21adc1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:15:04.897367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e55d-dad5-4e23-ad3b-ea806a21adc1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:15:04.901632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4b5cf-4344-4b95-8915-52a47d000a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:15:04.902275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4b5cf-4344-4b95-8915-52a47d000a30 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:15:04.902554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4b5cf-4344-4b95-8915-52a47d000a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:15:04.902825 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4b5cf-4344-4b95-8915-52a47d000a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:15:04.912049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4b5cf-4344-4b95-8915-52a47d000a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:15:04.913667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4b5cf-4344-4b95-8915-52a47d000a30 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:15:04.922912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 758/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:15:14.823832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=33 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:15:14.918933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=33 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:16:29.792302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e347d-2997-4347-9e66-4a6ace55fa6a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:29.792902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e347d-2997-4347-9e66-4a6ace55fa6a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:29.792902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e347d-2997-4347-9e66-4a6ace55fa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:29.812398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-302e347d-2997-4347-9e66-4a6ace55fa6a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:16:29.813595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-302e347d-2997-4347-9e66-4a6ace55fa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:16:29.823463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 736/1494] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:16:29.834043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e5ba-a352-4f6e-ab3f-43557f24b6fa None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:29.834855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e5ba-a352-4f6e-ab3f-43557f24b6fa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:29.834855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e5ba-a352-4f6e-ab3f-43557f24b6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:29.857089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8a11e5ba-a352-4f6e-ab3f-43557f24b6fa None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:16:29.858299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8a11e5ba-a352-4f6e-ab3f-43557f24b6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:16:29.867983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 759/1495] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:16:29.874586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e347d-2997-4347-9e66-4a6ace55fa6a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:29.885254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e347d-2997-4347-9e66-4a6ace55fa6a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:29.893411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:29.894437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:29.894727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:29.895060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:29.896215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:16:29.896626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:29.898166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:16:29.898166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jun 23 08:16:29.898166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: Jun 23 08:16:29.899202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:29.911673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 737/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:16:29.919789 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e5ba-a352-4f6e-ab3f-43557f24b6fa None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:29.929510 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e5ba-a352-4f6e-ab3f-43557f24b6fa None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:29.936415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:29.937066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:29.937344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:29.937636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:29.938337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:29.940167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:16:29.940167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jun 23 08:16:29.940167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: Jun 23 08:16:29.941350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:29.990968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 760/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:16:29 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:16:29.996389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:30.007087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdad811-bdac-4d62-8bd0-e7a5a88e7db4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:30.065126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:30.065720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.065999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.066273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.066962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:30.068040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:16:30.068040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jun 23 08:16:30.068040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: Jun 23 08:16:30.069244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:30.092725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 738/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:16:29 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:16:30.099012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:30.114705 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f2b46-55ae-4ce9-8f2d-9ad988523f3f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:30.120968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:30.121719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.122004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.122274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.123006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:30.124129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:16:30.124129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jun 23 08:16:30.124129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: Jun 23 08:16:30.125001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:30.137912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 761/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:16:30.141771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:30.151894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69ff0c-dcc5-4fc5-ab1e-3976159f7e92 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:30.157033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f036e2-b28a-4473-b054-495b65854415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:30.157738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f036e2-b28a-4473-b054-495b65854415 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.158022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f036e2-b28a-4473-b054-495b65854415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.158300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f036e2-b28a-4473-b054-495b65854415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.159040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f036e2-b28a-4473-b054-495b65854415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:30.160190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-81f036e2-b28a-4473-b054-495b65854415 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:16:30.160190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jun 23 08:16:30.160190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: Jun 23 08:16:30.161074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f036e2-b28a-4473-b054-495b65854415 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:30.175023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 739/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:16:30.179248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:30.190184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb9fad-f826-4093-806b-a57c1f8ba475 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:30.195382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed3f14-fbc0-454b-a191-d99f64ab78bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:30.196242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed3f14-fbc0-454b-a191-d99f64ab78bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.196524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed3f14-fbc0-454b-a191-d99f64ab78bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.196799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed3f14-fbc0-454b-a191-d99f64ab78bf admin admin] PATH_INFO: `/v3/projects/adba3d8a2e284c4793371b5c17a2eed4/users/10fb17c6ed2f472f92c446a76175d9ca/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.212694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed3f14-fbc0-454b-a191-d99f64ab78bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=adba3d8a2e284c4793371b5c17a2eed4, user_id=10fb17c6ed2f472f92c446a76175d9ca, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:30.213784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-78ed3f14-fbc0-454b-a191-d99f64ab78bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:16:30.213784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jun 23 08:16:30.213784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: Jun 23 08:16:30.215840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed3f14-fbc0-454b-a191-d99f64ab78bf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:30.233617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 762/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:16:30 2022] PUT /identity/v3/projects/adba3d8a2e284c4793371b5c17a2eed4/users/10fb17c6ed2f472f92c446a76175d9ca/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:16:30.246228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed392-3327-428e-9c8e-4ec6ee5cd26d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.246526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed392-3327-428e-9c8e-4ec6ee5cd26d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.246806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed392-3327-428e-9c8e-4ec6ee5cd26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.276055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-92eed392-3327-428e-9c8e-4ec6ee5cd26d None None] MFA Rules not processed for user `10fb17c6ed2f472f92c446a76175d9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:16:30.277272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-92eed392-3327-428e-9c8e-4ec6ee5cd26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:16:30.344737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 740/1502] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:16:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:16:30.359808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56417d92-3edf-4cbf-bfcc-6c9e9d27f12f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.360100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56417d92-3edf-4cbf-bfcc-6c9e9d27f12f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.360382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56417d92-3edf-4cbf-bfcc-6c9e9d27f12f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.361758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 763/1503] 10.222.0.33 () {54 vars in 963 bytes} [Thu Jun 23 08:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:16:30.366697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b481d-6de6-4fb1-8065-70a2f641c5af None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.366990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b481d-6de6-4fb1-8065-70a2f641c5af None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.367415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b481d-6de6-4fb1-8065-70a2f641c5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.385948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f41b481d-6de6-4fb1-8065-70a2f641c5af None None] MFA Rules not processed for user `419321fbd62c4d95a39cfd498b4a97d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:16:30.387196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f41b481d-6de6-4fb1-8065-70a2f641c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:16:30.441383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:16:30.451108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 741/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Jun 23 08:16:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:16:30.459067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56417d92-3edf-4cbf-bfcc-6c9e9d27f12f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:30.469885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56417d92-3edf-4cbf-bfcc-6c9e9d27f12f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:30.473562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a74761-0479-4398-8865-eacafb60636f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:30.474199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a74761-0479-4398-8865-eacafb60636f service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.474474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a74761-0479-4398-8865-eacafb60636f service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.474746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a74761-0479-4398-8865-eacafb60636f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.483126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a74761-0479-4398-8865-eacafb60636f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:30.484683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a74761-0479-4398-8865-eacafb60636f service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:30.493921 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 764/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:16:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:16:30.544615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84232466-8c6b-4617-8f0c-0dd13e0652ef None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.544894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84232466-8c6b-4617-8f0c-0dd13e0652ef None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.545200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84232466-8c6b-4617-8f0c-0dd13e0652ef None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.547173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 742/1506] 10.222.0.33 () {52 vars in 829 bytes} [Thu Jun 23 08:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:16:30.551075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a74761-0479-4398-8865-eacafb60636f service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:30.616924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a74761-0479-4398-8865-eacafb60636f service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:30.623141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:30.624070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.624386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.624663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] PATH_INFO: `/v3/users/10fb17c6ed2f472f92c446a76175d9ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.625952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10fb17c6ed2f472f92c446a76175d9ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:30.627024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:16:30.627024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jun 23 08:16:30.627024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: Jun 23 08:16:30.628272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:30.698483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10fb17c6ed2f472f92c446a76175d9ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:30.699265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10fb17c6ed2f472f92c446a76175d9ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:30.711734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10fb17c6ed2f472f92c446a76175d9ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:30.731657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10fb17c6ed2f472f92c446a76175d9ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:30.736021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:16:30.746216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10fb17c6ed2f472f92c446a76175d9ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:30.754197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 765/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:16:30 2022] DELETE /identity/v3/users/10fb17c6ed2f472f92c446a76175d9ca => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:16:30.758939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6476f5c-21b9-46b8-9c0b-cb411233195e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.759251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6476f5c-21b9-46b8-9c0b-cb411233195e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.759531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6476f5c-21b9-46b8-9c0b-cb411233195e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.777613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a6476f5c-21b9-46b8-9c0b-cb411233195e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:16:30.778857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a6476f5c-21b9-46b8-9c0b-cb411233195e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:16:30.846844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 743/1508] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:16:30.857107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:30.858753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:16:30.929465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e9dd4-67f4-4321-ac19-d6a8e7d18007 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:30.934165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c9aeb-c594-4bba-b110-9949e2d2d671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:30.934845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c9aeb-c594-4bba-b110-9949e2d2d671 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:30.935134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c9aeb-c594-4bba-b110-9949e2d2d671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:30.935445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c9aeb-c594-4bba-b110-9949e2d2d671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:30.945895 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c9aeb-c594-4bba-b110-9949e2d2d671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:30.947392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c9aeb-c594-4bba-b110-9949e2d2d671 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:30.957002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 766/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:16:31.334557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6476f5c-21b9-46b8-9c0b-cb411233195e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:16:31.336337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a6476f5c-21b9-46b8-9c0b-cb411233195e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:16:31.348270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6476f5c-21b9-46b8-9c0b-cb411233195e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:16:31.354101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:16:31.354488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:16:31.354764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:16:31.355033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] PATH_INFO: `/v3/projects/adba3d8a2e284c4793371b5c17a2eed4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:16:31.356342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=adba3d8a2e284c4793371b5c17a2eed4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:16:31.357383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:16:31.357383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jun 23 08:16:31.357383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: Jun 23 08:16:31.358614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:16:31.423001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adba3d8a2e284c4793371b5c17a2eed4', 'request_id': 'req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:31.429280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adba3d8a2e284c4793371b5c17a2eed4', 'request_id': 'req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:31.440762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adba3d8a2e284c4793371b5c17a2eed4', 'request_id': 'req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:31.460311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] The token cache is being invalidate because project adba3d8a2e284c4793371b5c17a2eed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:16:31.460529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:31.461316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f8eedafe-3740-4f1c-9190-6617c9c9fa1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:16:31.463305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 744/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:16:31 2022] DELETE /identity/v3/projects/adba3d8a2e284c4793371b5c17a2eed4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:16:40.952731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:16:41.456732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:18:10.386596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09485f-28d6-4666-80c7-2f09358d63ac None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:10.387329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09485f-28d6-4666-80c7-2f09358d63ac None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:10.387329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09485f-28d6-4666-80c7-2f09358d63ac None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:10.388554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:10.393545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f7761-49b6-4280-8d34-a775248c6870 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:10.394011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f7761-49b6-4280-8d34-a775248c6870 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:10.394115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f7761-49b6-4280-8d34-a775248c6870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:10.413978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5b0f7761-49b6-4280-8d34-a775248c6870 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:10.415296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5b0f7761-49b6-4280-8d34-a775248c6870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:10.477133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:10.632453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09485f-28d6-4666-80c7-2f09358d63ac None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:10.634552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3d09485f-28d6-4666-80c7-2f09358d63ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:10.717850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09485f-28d6-4666-80c7-2f09358d63ac None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:10.728175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc32df0-5c58-490d-93cf-60a5e3e560f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:10.729152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc32df0-5c58-490d-93cf-60a5e3e560f6 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:10.729521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc32df0-5c58-490d-93cf-60a5e3e560f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:10.729898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc32df0-5c58-490d-93cf-60a5e3e560f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:10.740403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc32df0-5c58-490d-93cf-60a5e3e560f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:10.742030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc32df0-5c58-490d-93cf-60a5e3e560f6 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:10.752403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 768/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:11.579563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2064469-d416-4e51-8e80-90e83d02acf0 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:11.580137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2064469-d416-4e51-8e80-90e83d02acf0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:11.580137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2064469-d416-4e51-8e80-90e83d02acf0 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:11.581509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:11.589339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dfd61-cfba-4dbf-8f62-35c3f4ce2dad None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:11.589772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dfd61-cfba-4dbf-8f62-35c3f4ce2dad None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:11.589886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dfd61-cfba-4dbf-8f62-35c3f4ce2dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:11.616363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-522dfd61-cfba-4dbf-8f62-35c3f4ce2dad None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:11.617652 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-522dfd61-cfba-4dbf-8f62-35c3f4ce2dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:11.627924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:18:11.793027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2064469-d416-4e51-8e80-90e83d02acf0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:11.815424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2064469-d416-4e51-8e80-90e83d02acf0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:11.821742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d3cac-7e1f-46f1-8565-93cd95c3b21b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:11.822392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d3cac-7e1f-46f1-8565-93cd95c3b21b service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:11.822673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d3cac-7e1f-46f1-8565-93cd95c3b21b service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:11.822949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d3cac-7e1f-46f1-8565-93cd95c3b21b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:11.832363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d3cac-7e1f-46f1-8565-93cd95c3b21b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:11.834053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d3cac-7e1f-46f1-8565-93cd95c3b21b service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:11.844393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 747/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:12.567599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c2ec8-9cd8-45bf-b45e-e0d1ed45bdfd None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:12.567856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c2ec8-9cd8-45bf-b45e-e0d1ed45bdfd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:12.568151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c2ec8-9cd8-45bf-b45e-e0d1ed45bdfd None None] PATH_INFO: `/v3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:12.569711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 770/1517] 10.222.0.36 () {54 vars in 925 bytes} [Thu Jun 23 08:18:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:18:12.576391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00db345-9204-498c-b03f-05cad0cbcc21 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:12.576700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00db345-9204-498c-b03f-05cad0cbcc21 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:12.577435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00db345-9204-498c-b03f-05cad0cbcc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:12.602480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d00db345-9204-498c-b03f-05cad0cbcc21 None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:12.603741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d00db345-9204-498c-b03f-05cad0cbcc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:12.672129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 748/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Jun 23 08:18:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:21.539841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347514-4eb6-4165-a001-a3aefb0e5635 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:21.540390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347514-4eb6-4165-a001-a3aefb0e5635 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:21.540390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347514-4eb6-4165-a001-a3aefb0e5635 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:21.542232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 771/1519] 10.222.0.33 () {54 vars in 960 bytes} [Thu Jun 23 08:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:21.550683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e6750-7816-4d4a-a043-7e122f569704 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:21.551227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e6750-7816-4d4a-a043-7e122f569704 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:21.551343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e6750-7816-4d4a-a043-7e122f569704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:21.573092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-dc5e6750-7816-4d4a-a043-7e122f569704 None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:21.574427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dc5e6750-7816-4d4a-a043-7e122f569704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:21.584457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 749/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Thu Jun 23 08:18:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:21.594686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67347514-4eb6-4165-a001-a3aefb0e5635 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:21.611271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67347514-4eb6-4165-a001-a3aefb0e5635 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:21.617502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097dfcd-ccec-4d6c-9dea-bce4c9daea1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:21.618148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097dfcd-ccec-4d6c-9dea-bce4c9daea1e service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:21.618436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097dfcd-ccec-4d6c-9dea-bce4c9daea1e service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:21.618718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097dfcd-ccec-4d6c-9dea-bce4c9daea1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:21.627860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097dfcd-ccec-4d6c-9dea-bce4c9daea1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:21.629462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097dfcd-ccec-4d6c-9dea-bce4c9daea1e service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:21.639364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 772/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:23.096901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e2022-dcd6-4c7a-b48b-e077b5a7cccf None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:23.097506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e2022-dcd6-4c7a-b48b-e077b5a7cccf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:23.097506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e2022-dcd6-4c7a-b48b-e077b5a7cccf None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:23.098898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:23.105793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9700a81-b377-48ae-94c3-e720c968d617 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:23.106285 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9700a81-b377-48ae-94c3-e720c968d617 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:23.106388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9700a81-b377-48ae-94c3-e720c968d617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:23.126515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b9700a81-b377-48ae-94c3-e720c968d617 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:23.127731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b9700a81-b377-48ae-94c3-e720c968d617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:23.137720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:23.290453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e2022-dcd6-4c7a-b48b-e077b5a7cccf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:23.307749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e2022-dcd6-4c7a-b48b-e077b5a7cccf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:23.313463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ef4c8-5042-4187-a33a-00ab84a15e5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:23.314102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef4c8-5042-4187-a33a-00ab84a15e5c service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:23.314379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef4c8-5042-4187-a33a-00ab84a15e5c service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:23.314675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef4c8-5042-4187-a33a-00ab84a15e5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:23.322860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ef4c8-5042-4187-a33a-00ab84a15e5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:23.324508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ef4c8-5042-4187-a33a-00ab84a15e5c service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:23.334455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 751/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:24.142832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cfe84-d3b4-46e4-b456-169ee0b95414 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:24.143481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cfe84-d3b4-46e4-b456-169ee0b95414 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:24.143481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cfe84-d3b4-46e4-b456-169ee0b95414 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:24.144792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 08:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:24.150517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57244727-d70a-4699-aac1-c76a1f0d8973 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:24.150984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57244727-d70a-4699-aac1-c76a1f0d8973 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:24.151101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57244727-d70a-4699-aac1-c76a1f0d8973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:24.173651 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-57244727-d70a-4699-aac1-c76a1f0d8973 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:24.174918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-57244727-d70a-4699-aac1-c76a1f0d8973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:24.183772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 08:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:24.347264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cfe84-d3b4-46e4-b456-169ee0b95414 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:24.367957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cfe84-d3b4-46e4-b456-169ee0b95414 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:24.378322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafccb9-fcc2-40a6-8e48-66ee3494f32c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:24.379284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafccb9-fcc2-40a6-8e48-66ee3494f32c service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:24.379634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafccb9-fcc2-40a6-8e48-66ee3494f32c service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:24.379994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafccb9-fcc2-40a6-8e48-66ee3494f32c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:24.392537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafccb9-fcc2-40a6-8e48-66ee3494f32c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:24.394226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafccb9-fcc2-40a6-8e48-66ee3494f32c service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:24.403888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 775/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:28.293634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695abc97-a7c0-4b77-8ada-4a5b2a3f6f8d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:28.294282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695abc97-a7c0-4b77-8ada-4a5b2a3f6f8d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:28.294282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695abc97-a7c0-4b77-8ada-4a5b2a3f6f8d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:28.295628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:28.302245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7211-36e2-46da-af82-27e3d86ef4fb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:28.302961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7211-36e2-46da-af82-27e3d86ef4fb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:28.303116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7211-36e2-46da-af82-27e3d86ef4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:28.326290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e8dc7211-36e2-46da-af82-27e3d86ef4fb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:28.327547 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e8dc7211-36e2-46da-af82-27e3d86ef4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:28.336526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:28.376077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695abc97-a7c0-4b77-8ada-4a5b2a3f6f8d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:28.378884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-695abc97-a7c0-4b77-8ada-4a5b2a3f6f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:28.446061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695abc97-a7c0-4b77-8ada-4a5b2a3f6f8d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:28.450757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201dbaf-7455-4967-999a-af6bc5044c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:28.451532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201dbaf-7455-4967-999a-af6bc5044c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:28.452206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201dbaf-7455-4967-999a-af6bc5044c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:28.452206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201dbaf-7455-4967-999a-af6bc5044c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:28.466344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201dbaf-7455-4967-999a-af6bc5044c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:28.468128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201dbaf-7455-4967-999a-af6bc5044c4b service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:28.473176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:18:28.478328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 754/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:30.582664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcc368-e20b-4355-bc9b-566d51bd2d6f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:30.583465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcc368-e20b-4355-bc9b-566d51bd2d6f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:30.583465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcc368-e20b-4355-bc9b-566d51bd2d6f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:30.585541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:18:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:30.591776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152c3d-360e-45f1-93bc-b1b74c7b02f3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:30.592497 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152c3d-360e-45f1-93bc-b1b74c7b02f3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:30.592621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152c3d-360e-45f1-93bc-b1b74c7b02f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:30.612339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d1152c3d-360e-45f1-93bc-b1b74c7b02f3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:30.613474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d1152c3d-360e-45f1-93bc-b1b74c7b02f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:30.621919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:30.659249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcc368-e20b-4355-bc9b-566d51bd2d6f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:30.672780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcc368-e20b-4355-bc9b-566d51bd2d6f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:30.677410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6c2f2-4202-40ec-8c5d-1e6ea383cd6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:30.678101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c2f2-4202-40ec-8c5d-1e6ea383cd6c service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:30.678378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c2f2-4202-40ec-8c5d-1e6ea383cd6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:30.678692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c2f2-4202-40ec-8c5d-1e6ea383cd6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:30.689510 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6c2f2-4202-40ec-8c5d-1e6ea383cd6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:30.691142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6c2f2-4202-40ec-8c5d-1e6ea383cd6c service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:30.701052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 778/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:34.106572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732c9d6-529d-4d0a-a9fc-4df7e31a194d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:34.107532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732c9d6-529d-4d0a-a9fc-4df7e31a194d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:34.107532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732c9d6-529d-4d0a-a9fc-4df7e31a194d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:34.109192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:18:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:34.114541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3504f-211e-4d96-b54f-484cadcf9e42 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:34.115002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3504f-211e-4d96-b54f-484cadcf9e42 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:34.115107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3504f-211e-4d96-b54f-484cadcf9e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:34.122260 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=62 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:18:34.136714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b7b3504f-211e-4d96-b54f-484cadcf9e42 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:34.137917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b7b3504f-211e-4d96-b54f-484cadcf9e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:34.146605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:34.445628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732c9d6-529d-4d0a-a9fc-4df7e31a194d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:34.448664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2732c9d6-529d-4d0a-a9fc-4df7e31a194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:34.514000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732c9d6-529d-4d0a-a9fc-4df7e31a194d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:34.519142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6217d7d-48b7-49cc-8d01-038d342f7c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:34.519815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6217d7d-48b7-49cc-8d01-038d342f7c38 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:34.520108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6217d7d-48b7-49cc-8d01-038d342f7c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:34.520381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6217d7d-48b7-49cc-8d01-038d342f7c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:34.530610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6217d7d-48b7-49cc-8d01-038d342f7c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:34.532328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6217d7d-48b7-49cc-8d01-038d342f7c38 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:34.542398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 757/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:36.592171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201dfd3-1c26-44ee-a7d2-b1e484ce4b62 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:36.593065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201dfd3-1c26-44ee-a7d2-b1e484ce4b62 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:36.593272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201dfd3-1c26-44ee-a7d2-b1e484ce4b62 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:36.595532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 08:18:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:18:36.601939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0e8d-be69-4779-81f0-b144ae72609e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:36.602370 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0e8d-be69-4779-81f0-b144ae72609e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:36.602527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0e8d-be69-4779-81f0-b144ae72609e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:36.630477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ec0f0e8d-be69-4779-81f0-b144ae72609e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:18:36.631806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ec0f0e8d-be69-4779-81f0-b144ae72609e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:36.643859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 08:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:18:36.946996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201dfd3-1c26-44ee-a7d2-b1e484ce4b62 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:18:36.949094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1201dfd3-1c26-44ee-a7d2-b1e484ce4b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:18:36.963193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201dfd3-1c26-44ee-a7d2-b1e484ce4b62 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:18:36.967971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed111f-7239-4206-8beb-cf35ac7f48dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:18:36.968526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed111f-7239-4206-8beb-cf35ac7f48dd service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:18:36.968850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed111f-7239-4206-8beb-cf35ac7f48dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:18:36.969180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed111f-7239-4206-8beb-cf35ac7f48dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:18:36.980513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed111f-7239-4206-8beb-cf35ac7f48dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:18:36.981966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed111f-7239-4206-8beb-cf35ac7f48dd service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:18:36.992043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 781/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:18:46.624631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=32 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:18:46.987609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:04.494082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c467-4733-4bee-94fb-1d245369e986 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.513716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c467-4733-4bee-94fb-1d245369e986 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.513716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c467-4733-4bee-94fb-1d245369e986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.524654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b1d7c467-4733-4bee-94fb-1d245369e986 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:04.526079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b1d7c467-4733-4bee-94fb-1d245369e986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:04.543477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 759/1540] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:04.554426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44a442-3883-4a6a-8227-fe9ac88a5aab None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.555489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44a442-3883-4a6a-8227-fe9ac88a5aab None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.556020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44a442-3883-4a6a-8227-fe9ac88a5aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.588511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fb44a442-3883-4a6a-8227-fe9ac88a5aab None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:04.590231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fb44a442-3883-4a6a-8227-fe9ac88a5aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:04.599185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 782/1541] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:04.604366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7c467-4733-4bee-94fb-1d245369e986 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:04.617709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7c467-4733-4bee-94fb-1d245369e986 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:04.624505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:04.625225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.625506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.625781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.626614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:04.626937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:04.628907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:04.651564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 760/1542] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:04.660097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44a442-3883-4a6a-8227-fe9ac88a5aab None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:04.670497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44a442-3883-4a6a-8227-fe9ac88a5aab None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:04.682843 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a23c0-e7a3-48d6-a75a-6f60fce47255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:04.683982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a23c0-e7a3-48d6-a75a-6f60fce47255 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.684500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a23c0-e7a3-48d6-a75a-6f60fce47255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.685009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a23c0-e7a3-48d6-a75a-6f60fce47255 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.686185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a23c0-e7a3-48d6-a75a-6f60fce47255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:04.688773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a23c0-e7a3-48d6-a75a-6f60fce47255 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:04.733772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 783/1543] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:04 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:04.738766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:04.755362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3469f-caaf-4ac1-b8b2-8403a1501579 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:04.831676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babeb96-b8a5-4211-858e-46dd2ac8a885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:04.832363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babeb96-b8a5-4211-858e-46dd2ac8a885 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.832641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babeb96-b8a5-4211-858e-46dd2ac8a885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.832910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babeb96-b8a5-4211-858e-46dd2ac8a885 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.833802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babeb96-b8a5-4211-858e-46dd2ac8a885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:04.835788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babeb96-b8a5-4211-858e-46dd2ac8a885 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:04.866998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 761/1544] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:04 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:04.872718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a23c0-e7a3-48d6-a75a-6f60fce47255 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:04.889287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a23c0-e7a3-48d6-a75a-6f60fce47255 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:04.899742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f4f3a-d4ab-4ce7-9236-3a22c1162a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:04.904672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f4f3a-d4ab-4ce7-9236-3a22c1162a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.905782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f4f3a-d4ab-4ce7-9236-3a22c1162a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.906317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f4f3a-d4ab-4ce7-9236-3a22c1162a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.907360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f4f3a-d4ab-4ce7-9236-3a22c1162a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:04.909341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f4f3a-d4ab-4ce7-9236-3a22c1162a54 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:04.923623 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa04e2-b083-449c-9710-5e99ec52ac4e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.923893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa04e2-b083-449c-9710-5e99ec52ac4e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.924173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa04e2-b083-449c-9710-5e99ec52ac4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.925630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 784/1545] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:04.930237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f4f3a-d4ab-4ce7-9236-3a22c1162a54 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:04.941570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f4f3a-d4ab-4ce7-9236-3a22c1162a54 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:04.956741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfa4ac-5ec9-4891-a210-a1ef652e8882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:04.957522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfa4ac-5ec9-4891-a210-a1ef652e8882 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.957813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfa4ac-5ec9-4891-a210-a1ef652e8882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.958095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfa4ac-5ec9-4891-a210-a1ef652e8882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.958835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfa4ac-5ec9-4891-a210-a1ef652e8882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:04.960506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfa4ac-5ec9-4891-a210-a1ef652e8882 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:04.972013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e2fa04e2-b083-449c-9710-5e99ec52ac4e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:04.973236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e2fa04e2-b083-449c-9710-5e99ec52ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:04.976714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 785/1546] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:04.978056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd40b1-c0bf-4082-8c2c-1b81822283c2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:04.978347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd40b1-c0bf-4082-8c2c-1b81822283c2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:04.978629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd40b1-c0bf-4082-8c2c-1b81822283c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:04.994955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 762/1547] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 23 08:20:04.995849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa04e2-b083-449c-9710-5e99ec52ac4e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.009581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fadd40b1-c0bf-4082-8c2c-1b81822283c2 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.010759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fadd40b1-c0bf-4082-8c2c-1b81822283c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.015310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa04e2-b083-449c-9710-5e99ec52ac4e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.023831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c5c50-3890-44a8-b3e5-1cd4adaf828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.024764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c5c50-3890-44a8-b3e5-1cd4adaf828e admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.025038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c5c50-3890-44a8-b3e5-1cd4adaf828e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.025306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c5c50-3890-44a8-b3e5-1cd4adaf828e admin admin] PATH_INFO: `/v3/projects/89774ed6e4964866bf6fd73e32dee267/users/9b178568bc5e4876bdab388ac3302f0a/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.028128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 786/1548] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.029548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219dadd-8ecf-4323-b974-2b7659ef4601 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.029836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219dadd-8ecf-4323-b974-2b7659ef4601 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.030170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219dadd-8ecf-4323-b974-2b7659ef4601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.065238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c5c50-3890-44a8-b3e5-1cd4adaf828e admin admin] RBAC: Authorizing `identity:create_grant(project_id=89774ed6e4964866bf6fd73e32dee267, user_id=9b178568bc5e4876bdab388ac3302f0a, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.066300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-d02c5c50-3890-44a8-b3e5-1cd4adaf828e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:20:05.066300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jun 23 08:20:05.066300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: Jun 23 08:20:05.068248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c5c50-3890-44a8-b3e5-1cd4adaf828e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.074673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d219dadd-8ecf-4323-b974-2b7659ef4601 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.075884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d219dadd-8ecf-4323-b974-2b7659ef4601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.094961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 763/1549] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:04 2022] PUT /identity/v3/projects/89774ed6e4964866bf6fd73e32dee267/users/9b178568bc5e4876bdab388ac3302f0a/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:05.097270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d1e56-c6fc-4b9c-adc3-19d260f463ef None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.097569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d1e56-c6fc-4b9c-adc3-19d260f463ef None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.097838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d1e56-c6fc-4b9c-adc3-19d260f463ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.098257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 787/1550] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.099810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72141718-32c6-4ee9-928a-61dccc043a15 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.100086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72141718-32c6-4ee9-928a-61dccc043a15 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.100408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72141718-32c6-4ee9-928a-61dccc043a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.129198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ba1d1e56-c6fc-4b9c-adc3-19d260f463ef None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.130417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ba1d1e56-c6fc-4b9c-adc3-19d260f463ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.130564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-72141718-32c6-4ee9-928a-61dccc043a15 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.131827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-72141718-32c6-4ee9-928a-61dccc043a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.192063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 764/1551] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.193462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997f50-b881-44b0-8b68-2462b13b6c2e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.193753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997f50-b881-44b0-8b68-2462b13b6c2e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.194028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997f50-b881-44b0-8b68-2462b13b6c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.203186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 788/1552] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.204623 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330839c-778c-4216-8057-36f29d919bbb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.204916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330839c-778c-4216-8057-36f29d919bbb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.205196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330839c-778c-4216-8057-36f29d919bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.215583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b0997f50-b881-44b0-8b68-2462b13b6c2e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.216790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b0997f50-b881-44b0-8b68-2462b13b6c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.222100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5330839c-778c-4216-8057-36f29d919bbb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.223305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5330839c-778c-4216-8057-36f29d919bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.225447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 765/1553] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.227479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660fd40-76df-405a-8899-3e04d7cbb0dc None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.227773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660fd40-76df-405a-8899-3e04d7cbb0dc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.228074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660fd40-76df-405a-8899-3e04d7cbb0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.232655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 789/1554] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.234339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e2ae7-d8fb-4714-a4db-a5cb1dfaa2f8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.234647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e2ae7-d8fb-4714-a4db-a5cb1dfaa2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.234927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e2ae7-d8fb-4714-a4db-a5cb1dfaa2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.245949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f660fd40-76df-405a-8899-3e04d7cbb0dc None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.247116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f660fd40-76df-405a-8899-3e04d7cbb0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.255280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e50e2ae7-d8fb-4714-a4db-a5cb1dfaa2f8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.255871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 766/1555] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.256450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e50e2ae7-d8fb-4714-a4db-a5cb1dfaa2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.257166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60f427-f62d-4827-8d34-5dbca43e1fd8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.257474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60f427-f62d-4827-8d34-5dbca43e1fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.257888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60f427-f62d-4827-8d34-5dbca43e1fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.264828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 790/1556] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.266005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f2f8-81f6-41c4-b939-29d7e4f99d0c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.266511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f2f8-81f6-41c4-b939-29d7e4f99d0c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.266797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f2f8-81f6-41c4-b939-29d7e4f99d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.282982 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7d60f427-f62d-4827-8d34-5dbca43e1fd8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.284216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7d60f427-f62d-4827-8d34-5dbca43e1fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.289594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9e13f2f8-81f6-41c4-b939-29d7e4f99d0c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.290824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9e13f2f8-81f6-41c4-b939-29d7e4f99d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.293359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 767/1557] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.294626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d651-8d0a-42ca-94b8-d0954e7f1e7e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.294917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d651-8d0a-42ca-94b8-d0954e7f1e7e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.295177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d651-8d0a-42ca-94b8-d0954e7f1e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.305774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 791/1558] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.305774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f2f8-81f6-41c4-b939-29d7e4f99d0c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.311058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f2f8-81f6-41c4-b939-29d7e4f99d0c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.315842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.316515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.316793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.317071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.317800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:05.318104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.319129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c3c2d651-8d0a-42ca-94b8-d0954e7f1e7e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.319267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:20:05.319267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jun 23 08:20:05.319267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: Jun 23 08:20:05.320051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.320538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c3c2d651-8d0a-42ca-94b8-d0954e7f1e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.329198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 768/1559] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.330107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2d651-8d0a-42ca-94b8-d0954e7f1e7e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.335248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 792/1560] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:05.335248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.335248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.344120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2d651-8d0a-42ca-94b8-d0954e7f1e7e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.348730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.349395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.349679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.349961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.350693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:05.351009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.352697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.362120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:05.366079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 769/1561] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:05.367490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.377789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de271c-f523-4f7c-8b89-1134695d9ea3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.382399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9ba7a-441b-4513-9034-6a5aa060814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.383041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ba7a-441b-4513-9034-6a5aa060814c admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.383361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ba7a-441b-4513-9034-6a5aa060814c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.383638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ba7a-441b-4513-9034-6a5aa060814c admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.384359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9ba7a-441b-4513-9034-6a5aa060814c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:05.384794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9ba7a-441b-4513-9034-6a5aa060814c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.386395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9ba7a-441b-4513-9034-6a5aa060814c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.393735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e70d-b9f7-425b-8186-73bbb036d001 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.397399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8060d-bf8e-4ae5-8ac2-82601b4cc601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.398038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8060d-bf8e-4ae5-8ac2-82601b4cc601 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.398321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8060d-bf8e-4ae5-8ac2-82601b4cc601 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.398594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8060d-bf8e-4ae5-8ac2-82601b4cc601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.399397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 770/1562] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:05.400534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b372b0-5476-4280-8b3c-d7298dc2798a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.400820 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b372b0-5476-4280-8b3c-d7298dc2798a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.401095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b372b0-5476-4280-8b3c-d7298dc2798a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.413586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8060d-bf8e-4ae5-8ac2-82601b4cc601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.415187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8060d-bf8e-4ae5-8ac2-82601b4cc601 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.426146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 793/1563] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:05.427362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1ff05-6485-4c80-9a5c-00c17bce3799 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.427654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1ff05-6485-4c80-9a5c-00c17bce3799 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.427931 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1ff05-6485-4c80-9a5c-00c17bce3799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.444050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:05.452850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-10b1ff05-6485-4c80-9a5c-00c17bce3799 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.455369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10b1ff05-6485-4c80-9a5c-00c17bce3799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.465081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 794/1564] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.469573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750af07-1a7a-4b80-ab62-b7ceee279cf3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.469573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750af07-1a7a-4b80-ab62-b7ceee279cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.469573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750af07-1a7a-4b80-ab62-b7ceee279cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.492574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4750af07-1a7a-4b80-ab62-b7ceee279cf3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.493795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4750af07-1a7a-4b80-ab62-b7ceee279cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.502382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 795/1565] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.509675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750af07-1a7a-4b80-ab62-b7ceee279cf3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.522125 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750af07-1a7a-4b80-ab62-b7ceee279cf3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.527360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.528261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.528545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.528818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.529700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:05.530209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.531998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.543994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f2b372b0-5476-4280-8b3c-d7298dc2798a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:05.545235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f2b372b0-5476-4280-8b3c-d7298dc2798a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:05.546237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 796/1566] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:05.552443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.554128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 771/1567] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:05.554930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b372b0-5476-4280-8b3c-d7298dc2798a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.565410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b372b0-5476-4280-8b3c-d7298dc2798a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.566210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e099-f43f-42cc-8378-02ca5aaab123 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.570535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.571276 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.571594 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.571826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124595f-3d84-440a-b8ab-39557357e3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.571977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.572653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.572822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124595f-3d84-440a-b8ab-39557357e3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.573187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124595f-3d84-440a-b8ab-39557357e3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.573573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124595f-3d84-440a-b8ab-39557357e3d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.573865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:20:05.573865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jun 23 08:20:05.573865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: Jun 23 08:20:05.574588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124595f-3d84-440a-b8ab-39557357e3d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.575047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.577075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124595f-3d84-440a-b8ab-39557357e3d0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.604334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 772/1568] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:05.610915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 797/1569] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:05.611103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.611263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124595f-3d84-440a-b8ab-39557357e3d0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.688241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124595f-3d84-440a-b8ab-39557357e3d0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.691109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ec33-0186-4638-98b9-0f3ca574a0a1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.693221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.693941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.694225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.694507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.695262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:05.695577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.697240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.697488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343b05c-3925-4887-9287-ad64beef447f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.698311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343b05c-3925-4887-9287-ad64beef447f admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.698658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343b05c-3925-4887-9287-ad64beef447f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.699297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343b05c-3925-4887-9287-ad64beef447f admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.700605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343b05c-3925-4887-9287-ad64beef447f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.703904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343b05c-3925-4887-9287-ad64beef447f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.717401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 798/1570] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:05.717401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.730892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064339c-db63-4719-a84c-36edae7e5b24 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.736668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 773/1571] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:05.737199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.738021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.738367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.738715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.739568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:05.740157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.740595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343b05c-3925-4887-9287-ad64beef447f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.741800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.755676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 799/1572] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:05.763593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.782862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee939f89-5c3a-42c2-a65e-9ca3fe24ebdd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.824438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343b05c-3925-4887-9287-ad64beef447f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.833927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0474d-46f3-46f8-b29a-26e2c00fdd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.835527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0474d-46f3-46f8-b29a-26e2c00fdd5d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.835979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0474d-46f3-46f8-b29a-26e2c00fdd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.836416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0474d-46f3-46f8-b29a-26e2c00fdd5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.837589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0474d-46f3-46f8-b29a-26e2c00fdd5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.842205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0474d-46f3-46f8-b29a-26e2c00fdd5d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.855787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.856739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.857029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.857357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.858391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:05.858897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.860907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.870923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 774/1573] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:05.882659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 800/1574] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:05.882659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.882974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0474d-46f3-46f8-b29a-26e2c00fdd5d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.895136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066b11-9c69-4c63-b6bc-c9535d097d29 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.917492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:05.931479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:05.957802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0474d-46f3-46f8-b29a-26e2c00fdd5d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:05.963046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b9f71-8ada-476c-ad3f-56a65d059c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.963247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:05.963756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b9f71-8ada-476c-ad3f-56a65d059c7d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.963883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:05.964054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b9f71-8ada-476c-ad3f-56a65d059c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.964248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:05.964352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b9f71-8ada-476c-ad3f-56a65d059c7d admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.964532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:05.964985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b9f71-8ada-476c-ad3f-56a65d059c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.965247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:05.966337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:20:05.966337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jun 23 08:20:05.966337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: Jun 23 08:20:05.967094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b9f71-8ada-476c-ad3f-56a65d059c7d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.968043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:05.990000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 775/1575] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:05.990812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b9f71-8ada-476c-ad3f-56a65d059c7d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:05.999413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 801/1576] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.000643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.003256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b9f71-8ada-476c-ad3f-56a65d059c7d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.007967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bf17a-9544-4e27-887d-28519c1eaaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.008694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bf17a-9544-4e27-887d-28519c1eaaa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.009049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bf17a-9544-4e27-887d-28519c1eaaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.009391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bf17a-9544-4e27-887d-28519c1eaaa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.010151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bf17a-9544-4e27-887d-28519c1eaaa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.011117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dc759-2b23-4b8f-8886-69321f153a8d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.012224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bf17a-9544-4e27-887d-28519c1eaaa0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.016159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b3d46-9d0f-4047-8e48-c773ba39ba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.016898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b3d46-9d0f-4047-8e48-c773ba39ba00 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.017230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b3d46-9d0f-4047-8e48-c773ba39ba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.017702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b3d46-9d0f-4047-8e48-c773ba39ba00 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.018458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b3d46-9d0f-4047-8e48-c773ba39ba00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.020545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b3d46-9d0f-4047-8e48-c773ba39ba00 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.043647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 776/1577] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.044789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bf17a-9544-4e27-887d-28519c1eaaa0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.044911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 802/1578] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:05 2022] POST /identity/v3/users => generated 282 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.045820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b3d46-9d0f-4047-8e48-c773ba39ba00 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.057840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b3d46-9d0f-4047-8e48-c773ba39ba00 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.125134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bf17a-9544-4e27-887d-28519c1eaaa0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.129809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64227-cd62-40c5-9377-00602c331561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.130090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e9f9-e11c-47e9-9df4-8a0f72e0a3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.130489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64227-cd62-40c5-9377-00602c331561 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.130774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e9f9-e11c-47e9-9df4-8a0f72e0a3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.130878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64227-cd62-40c5-9377-00602c331561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.131056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64227-cd62-40c5-9377-00602c331561 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.131263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e9f9-e11c-47e9-9df4-8a0f72e0a3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.131621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e9f9-e11c-47e9-9df4-8a0f72e0a3d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.131796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa64227-cd62-40c5-9377-00602c331561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.132331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e9f9-e11c-47e9-9df4-8a0f72e0a3d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.133964 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa64227-cd62-40c5-9377-00602c331561 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.134295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e9f9-e11c-47e9-9df4-8a0f72e0a3d6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.160326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 803/1579] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:06 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.161604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64227-cd62-40c5-9377-00602c331561 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.164645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 777/1580] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:06 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.165700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e9f9-e11c-47e9-9df4-8a0f72e0a3d6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.174148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64227-cd62-40c5-9377-00602c331561 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.254420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870bd2a-0ef2-4bf2-aff9-6f2df019369a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.256061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870bd2a-0ef2-4bf2-aff9-6f2df019369a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.256549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870bd2a-0ef2-4bf2-aff9-6f2df019369a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.256927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870bd2a-0ef2-4bf2-aff9-6f2df019369a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.258792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870bd2a-0ef2-4bf2-aff9-6f2df019369a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.261119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e9f9-e11c-47e9-9df4-8a0f72e0a3d6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.261642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870bd2a-0ef2-4bf2-aff9-6f2df019369a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.266643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7f0e9-aedd-4ecf-ad79-b8bfd419bfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.267750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7f0e9-aedd-4ecf-ad79-b8bfd419bfdb admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.268046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7f0e9-aedd-4ecf-ad79-b8bfd419bfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.268284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7f0e9-aedd-4ecf-ad79-b8bfd419bfdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.269155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7f0e9-aedd-4ecf-ad79-b8bfd419bfdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.271002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7f0e9-aedd-4ecf-ad79-b8bfd419bfdb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.286670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 778/1581] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.286670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7f0e9-aedd-4ecf-ad79-b8bfd419bfdb admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.297008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7f0e9-aedd-4ecf-ad79-b8bfd419bfdb admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.301799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df4925-628e-45af-8783-8d50618c563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.302607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df4925-628e-45af-8783-8d50618c563a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.302951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df4925-628e-45af-8783-8d50618c563a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.303500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df4925-628e-45af-8783-8d50618c563a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.304306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df4925-628e-45af-8783-8d50618c563a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.306000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df4925-628e-45af-8783-8d50618c563a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.307429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 804/1582] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:06 2022] POST /identity/v3/projects => generated 393 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.308507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870bd2a-0ef2-4bf2-aff9-6f2df019369a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.319073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 779/1583] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.320017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df4925-628e-45af-8783-8d50618c563a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.331521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df4925-628e-45af-8783-8d50618c563a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.335789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:06.336996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:06.391523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870bd2a-0ef2-4bf2-aff9-6f2df019369a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.398248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056f9fd-1355-4128-8319-2b3a786457e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.399678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f9fd-1355-4128-8319-2b3a786457e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.400141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f9fd-1355-4128-8319-2b3a786457e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.400492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f9fd-1355-4128-8319-2b3a786457e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.401538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056f9fd-1355-4128-8319-2b3a786457e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.404045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056f9fd-1355-4128-8319-2b3a786457e9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.411689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af1b9b-9df2-4415-b59c-5b96240facd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.413435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af1b9b-9df2-4415-b59c-5b96240facd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.413832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af1b9b-9df2-4415-b59c-5b96240facd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.414205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af1b9b-9df2-4415-b59c-5b96240facd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.415775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af1b9b-9df2-4415-b59c-5b96240facd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.416109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 805/1584] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.418910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af1b9b-9df2-4415-b59c-5b96240facd5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.420477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056f9fd-1355-4128-8319-2b3a786457e9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.433492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056f9fd-1355-4128-8319-2b3a786457e9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.439428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450a64a-dba6-4e0b-8cce-06ee7d73aca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.440258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450a64a-dba6-4e0b-8cce-06ee7d73aca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.440699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450a64a-dba6-4e0b-8cce-06ee7d73aca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.440982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450a64a-dba6-4e0b-8cce-06ee7d73aca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.441720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450a64a-dba6-4e0b-8cce-06ee7d73aca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.443422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450a64a-dba6-4e0b-8cce-06ee7d73aca5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.448316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 780/1585] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:06 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.449335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af1b9b-9df2-4415-b59c-5b96240facd5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.459743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 806/1586] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.460710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450a64a-dba6-4e0b-8cce-06ee7d73aca5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.466188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af1b9b-9df2-4415-b59c-5b96240facd5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.472428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a0beb-227d-4f08-9162-e139e69ccbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.472819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450a64a-dba6-4e0b-8cce-06ee7d73aca5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.473493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a0beb-227d-4f08-9162-e139e69ccbaa admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.473863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a0beb-227d-4f08-9162-e139e69ccbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.474210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a0beb-227d-4f08-9162-e139e69ccbaa admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.475053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a0beb-227d-4f08-9162-e139e69ccbaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.477929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a0beb-227d-4f08-9162-e139e69ccbaa admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.480915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f749d7-c8ec-4419-94c1-11cedafa3f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.482291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f749d7-c8ec-4419-94c1-11cedafa3f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.482735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f749d7-c8ec-4419-94c1-11cedafa3f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.483333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f749d7-c8ec-4419-94c1-11cedafa3f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.484296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f749d7-c8ec-4419-94c1-11cedafa3f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.486290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f749d7-c8ec-4419-94c1-11cedafa3f65 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.501369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 807/1587] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.503197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f749d7-c8ec-4419-94c1-11cedafa3f65 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.512671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 781/1588] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:06 2022] POST /identity/v3/users => generated 290 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.513919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a0beb-227d-4f08-9162-e139e69ccbaa admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.519727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f749d7-c8ec-4419-94c1-11cedafa3f65 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.526028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a0beb-227d-4f08-9162-e139e69ccbaa admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.526901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d589b-39d8-48e4-b3b4-29047bdf7f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.528003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d589b-39d8-48e4-b3b4-29047bdf7f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.528458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d589b-39d8-48e4-b3b4-29047bdf7f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.528982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d589b-39d8-48e4-b3b4-29047bdf7f1f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.529909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d589b-39d8-48e4-b3b4-29047bdf7f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.531706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9750be-76c5-46df-bf5b-3eabf9ff4315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.532642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9750be-76c5-46df-bf5b-3eabf9ff4315 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.532981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d589b-39d8-48e4-b3b4-29047bdf7f1f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.533119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9750be-76c5-46df-bf5b-3eabf9ff4315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.533443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9750be-76c5-46df-bf5b-3eabf9ff4315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.534428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9750be-76c5-46df-bf5b-3eabf9ff4315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.536271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9750be-76c5-46df-bf5b-3eabf9ff4315 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.553866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 782/1589] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.553866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9750be-76c5-46df-bf5b-3eabf9ff4315 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.563462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9750be-76c5-46df-bf5b-3eabf9ff4315 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.566892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 808/1590] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:06 2022] POST /identity/v3/users => generated 297 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:06.568729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d589b-39d8-48e4-b3b4-29047bdf7f1f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.571699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ff978-0ee3-4ead-977f-c1a6e4887203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.572551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ff978-0ee3-4ead-977f-c1a6e4887203 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.573047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ff978-0ee3-4ead-977f-c1a6e4887203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.573433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ff978-0ee3-4ead-977f-c1a6e4887203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.574255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ff978-0ee3-4ead-977f-c1a6e4887203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.576074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ff978-0ee3-4ead-977f-c1a6e4887203 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.582622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d589b-39d8-48e4-b3b4-29047bdf7f1f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.591060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 783/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.591060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ff978-0ee3-4ead-977f-c1a6e4887203 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.593795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042ed2a-cbd2-40e9-823d-85b8dcc21bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.594575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ed2a-cbd2-40e9-823d-85b8dcc21bce admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.594861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ed2a-cbd2-40e9-823d-85b8dcc21bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.595133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ed2a-cbd2-40e9-823d-85b8dcc21bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.595904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042ed2a-cbd2-40e9-823d-85b8dcc21bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.600570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042ed2a-cbd2-40e9-823d-85b8dcc21bce admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.602307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ff978-0ee3-4ead-977f-c1a6e4887203 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.609588 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d8616-d60c-4b26-9abe-772cb7183d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.610424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d8616-d60c-4b26-9abe-772cb7183d9e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.610803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d8616-d60c-4b26-9abe-772cb7183d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.611170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d8616-d60c-4b26-9abe-772cb7183d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.612043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d8616-d60c-4b26-9abe-772cb7183d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.613737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d8616-d60c-4b26-9abe-772cb7183d9e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.618727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 809/1592] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.620238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042ed2a-cbd2-40e9-823d-85b8dcc21bce admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.628952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 784/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.632269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d8616-d60c-4b26-9abe-772cb7183d9e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.633240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042ed2a-cbd2-40e9-823d-85b8dcc21bce admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.638603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a6087-02a3-482b-9ae6-7c0f19973ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.639942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a6087-02a3-482b-9ae6-7c0f19973ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.640316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a6087-02a3-482b-9ae6-7c0f19973ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.640616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a6087-02a3-482b-9ae6-7c0f19973ab2 admin admin] PATH_INFO: `/v3/projects/1c8a0a9fab3e42a58d3cb273c536922c/users/851d6918e1e64933b9bcacc4be364c76/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.650234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d8616-d60c-4b26-9abe-772cb7183d9e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.656887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c89a9-7294-49ac-8e7e-bba87b615cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.657388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c89a9-7294-49ac-8e7e-bba87b615cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.657778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c89a9-7294-49ac-8e7e-bba87b615cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.658131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c89a9-7294-49ac-8e7e-bba87b615cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.659692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c89a9-7294-49ac-8e7e-bba87b615cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.660587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a6087-02a3-482b-9ae6-7c0f19973ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c8a0a9fab3e42a58d3cb273c536922c, user_id=851d6918e1e64933b9bcacc4be364c76, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.663081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c89a9-7294-49ac-8e7e-bba87b615cfc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.667340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a6087-02a3-482b-9ae6-7c0f19973ab2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.676330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 785/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.677473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c89a9-7294-49ac-8e7e-bba87b615cfc admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.688292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c89a9-7294-49ac-8e7e-bba87b615cfc admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.690361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 810/1595] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:06 2022] PUT /identity/v3/projects/1c8a0a9fab3e42a58d3cb273c536922c/users/851d6918e1e64933b9bcacc4be364c76/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:06.691404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a6087-02a3-482b-9ae6-7c0f19973ab2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.776804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be700186-11b6-47f9-aa3e-33c9caea02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.777824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be700186-11b6-47f9-aa3e-33c9caea02ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.778161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be700186-11b6-47f9-aa3e-33c9caea02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.778493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be700186-11b6-47f9-aa3e-33c9caea02ac admin admin] PATH_INFO: `/v3/projects/007ea29822834890b7bdee37392945ef/users/0351e1ed24a74a309dbc26588b39b2d8/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.788315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a6087-02a3-482b-9ae6-7c0f19973ab2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.793865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82681892-bc71-47f1-ab50-f0c3ef240e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.794828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82681892-bc71-47f1-ab50-f0c3ef240e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.795190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82681892-bc71-47f1-ab50-f0c3ef240e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.795823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82681892-bc71-47f1-ab50-f0c3ef240e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.796667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82681892-bc71-47f1-ab50-f0c3ef240e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.798385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82681892-bc71-47f1-ab50-f0c3ef240e03 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.798664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be700186-11b6-47f9-aa3e-33c9caea02ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=007ea29822834890b7bdee37392945ef, user_id=0351e1ed24a74a309dbc26588b39b2d8, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.802527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be700186-11b6-47f9-aa3e-33c9caea02ac admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.810958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 811/1596] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.812017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82681892-bc71-47f1-ab50-f0c3ef240e03 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.823452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82681892-bc71-47f1-ab50-f0c3ef240e03 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.830120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 786/1597] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:06 2022] PUT /identity/v3/projects/007ea29822834890b7bdee37392945ef/users/0351e1ed24a74a309dbc26588b39b2d8/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:06.831423 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be700186-11b6-47f9-aa3e-33c9caea02ac admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.876363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:06.878100 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:06.919038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a95f2-e247-411e-924d-b0d02eb60cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.920144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a95f2-e247-411e-924d-b0d02eb60cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.920545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a95f2-e247-411e-924d-b0d02eb60cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.921005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a95f2-e247-411e-924d-b0d02eb60cd0 admin admin] PATH_INFO: `/v3/projects/718f03eb90184025a738418782f8f7b3/users/1ba291bab038457a9d3fa215a2231e54/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.934972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be700186-11b6-47f9-aa3e-33c9caea02ac admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:06.940786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebd416-2bb5-4ba4-b3b8-fb980a5664e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:06.941116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a95f2-e247-411e-924d-b0d02eb60cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718f03eb90184025a738418782f8f7b3, user_id=1ba291bab038457a9d3fa215a2231e54, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.941988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebd416-2bb5-4ba4-b3b8-fb980a5664e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:06.942416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebd416-2bb5-4ba4-b3b8-fb980a5664e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:06.942895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebd416-2bb5-4ba4-b3b8-fb980a5664e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:06.944072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebd416-2bb5-4ba4-b3b8-fb980a5664e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:06.944828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a95f2-e247-411e-924d-b0d02eb60cd0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.946362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebd416-2bb5-4ba4-b3b8-fb980a5664e1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:06.963531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 787/1598] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:06.964828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebd416-2bb5-4ba4-b3b8-fb980a5664e1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:06.971259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 812/1599] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:06 2022] PUT /identity/v3/projects/718f03eb90184025a738418782f8f7b3/users/1ba291bab038457a9d3fa215a2231e54/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:06.971259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a95f2-e247-411e-924d-b0d02eb60cd0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:07.011991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebd416-2bb5-4ba4-b3b8-fb980a5664e1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.085211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a95f2-e247-411e-924d-b0d02eb60cd0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.089868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8e1e7-c2d4-4198-bbea-32ae53cdf7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.090543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8e1e7-c2d4-4198-bbea-32ae53cdf7dc admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.090776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f62ed4-3a31-4892-96df-0688c66106fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.090893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8e1e7-c2d4-4198-bbea-32ae53cdf7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.091080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8e1e7-c2d4-4198-bbea-32ae53cdf7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.091837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8e1e7-c2d4-4198-bbea-32ae53cdf7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.093051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f62ed4-3a31-4892-96df-0688c66106fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.093589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8e1e7-c2d4-4198-bbea-32ae53cdf7dc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.093920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f62ed4-3a31-4892-96df-0688c66106fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.096042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f62ed4-3a31-4892-96df-0688c66106fb admin admin] PATH_INFO: `/v3/projects/e5c2f087908e492f8874200a56048571/users/5d4550977e4b47738deb7747899ef5e6/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.117944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 813/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:07.117944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d469b-9acf-4194-8eaf-354efcd7e9ec None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.117944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d469b-9acf-4194-8eaf-354efcd7e9ec None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.117944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d469b-9acf-4194-8eaf-354efcd7e9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.118613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f62ed4-3a31-4892-96df-0688c66106fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c2f087908e492f8874200a56048571, user_id=5d4550977e4b47738deb7747899ef5e6, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.122116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f62ed4-3a31-4892-96df-0688c66106fb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.146245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 788/1601] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:06 2022] PUT /identity/v3/projects/e5c2f087908e492f8874200a56048571/users/5d4550977e4b47738deb7747899ef5e6/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:07.147459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f62ed4-3a31-4892-96df-0688c66106fb admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:07.158733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d78d469b-9acf-4194-8eaf-354efcd7e9ec None None] MFA Rules not processed for user `851d6918e1e64933b9bcacc4be364c76`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:07.160625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d78d469b-9acf-4194-8eaf-354efcd7e9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:07.238034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f62ed4-3a31-4892-96df-0688c66106fb admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.242716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b2879-590a-403b-9c8b-4ee269e1fcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.243770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b2879-590a-403b-9c8b-4ee269e1fcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.243770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b2879-590a-403b-9c8b-4ee269e1fcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.244088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b2879-590a-403b-9c8b-4ee269e1fcc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.244716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b2879-590a-403b-9c8b-4ee269e1fcc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.246327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b2879-590a-403b-9c8b-4ee269e1fcc7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.259299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 789/1602] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:07.259912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340835c8-78dd-4f20-ae39-77b4ae0b8b31 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.260192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340835c8-78dd-4f20-ae39-77b4ae0b8b31 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.260465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340835c8-78dd-4f20-ae39-77b4ae0b8b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.264807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 814/1603] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:07.379029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d469b-9acf-4194-8eaf-354efcd7e9ec None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:07.379029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d469b-9acf-4194-8eaf-354efcd7e9ec None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.379029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4642c-fbc2-4203-ad50-f0d107fafdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.379029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4642c-fbc2-4203-ad50-f0d107fafdae admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.379937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4642c-fbc2-4203-ad50-f0d107fafdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.379937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4642c-fbc2-4203-ad50-f0d107fafdae admin admin] PATH_INFO: `/v3/projects/4979d926d8894e3db358f16bd6bf7621/users/938efacddf794fd18cd323043906b6c1/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.380543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-340835c8-78dd-4f20-ae39-77b4ae0b8b31 None None] MFA Rules not processed for user `0351e1ed24a74a309dbc26588b39b2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:07.380543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-340835c8-78dd-4f20-ae39-77b4ae0b8b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:07.380906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4642c-fbc2-4203-ad50-f0d107fafdae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4979d926d8894e3db358f16bd6bf7621, user_id=938efacddf794fd18cd323043906b6c1, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.380906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4642c-fbc2-4203-ad50-f0d107fafdae admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.396544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:07.404307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 815/1604] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:07 2022] PUT /identity/v3/projects/4979d926d8894e3db358f16bd6bf7621/users/938efacddf794fd18cd323043906b6c1/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:07.404307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e56f-0b15-43eb-9648-9f76896cc7d7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.404307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e56f-0b15-43eb-9648-9f76896cc7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.404865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e56f-0b15-43eb-9648-9f76896cc7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.462686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7a22e56f-0b15-43eb-9648-9f76896cc7d7 None None] MFA Rules not processed for user `1ba291bab038457a9d3fa215a2231e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:07.463999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7a22e56f-0b15-43eb-9648-9f76896cc7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:07.475964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 790/1605] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:07.475964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340835c8-78dd-4f20-ae39-77b4ae0b8b31 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:07.483599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:07.497825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340835c8-78dd-4f20-ae39-77b4ae0b8b31 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.567899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 816/1606] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:07.569412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69369c4d-e7e9-498f-8d03-b9a509e26b00 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.569412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69369c4d-e7e9-498f-8d03-b9a509e26b00 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.569740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69369c4d-e7e9-498f-8d03-b9a509e26b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.601191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b092c-1c6b-4343-a1e7-dc07c2a0271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.602740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b092c-1c6b-4343-a1e7-dc07c2a0271c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.603434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-69369c4d-e7e9-498f-8d03-b9a509e26b00 None None] MFA Rules not processed for user `5d4550977e4b47738deb7747899ef5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:07.603775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b092c-1c6b-4343-a1e7-dc07c2a0271c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.604129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b092c-1c6b-4343-a1e7-dc07c2a0271c admin admin] PATH_INFO: `/v3/projects/0e2534820f9840f5b45a08bb2428be70/users/eaf926af634c4ec2ada5fc8f4aa95e07/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.604910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-69369c4d-e7e9-498f-8d03-b9a509e26b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:07.628665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b092c-1c6b-4343-a1e7-dc07c2a0271c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2534820f9840f5b45a08bb2428be70, user_id=eaf926af634c4ec2ada5fc8f4aa95e07, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.631931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b092c-1c6b-4343-a1e7-dc07c2a0271c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.650738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 791/1607] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:07 2022] PUT /identity/v3/projects/0e2534820f9840f5b45a08bb2428be70/users/eaf926af634c4ec2ada5fc8f4aa95e07/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:07.651848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b092c-1c6b-4343-a1e7-dc07c2a0271c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:07.700398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 817/1608] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:07.702572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5781ec-6f91-46aa-a457-3d8cfa0883c2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.703032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5781ec-6f91-46aa-a457-3d8cfa0883c2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.703531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5781ec-6f91-46aa-a457-3d8cfa0883c2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.705061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 818/1609] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 23 08:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:07.706542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77317a-4621-4655-b2a8-ae0bd8ebd3bd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.707026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77317a-4621-4655-b2a8-ae0bd8ebd3bd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.707562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77317a-4621-4655-b2a8-ae0bd8ebd3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.727971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4f77317a-4621-4655-b2a8-ae0bd8ebd3bd None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:07.730218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4f77317a-4621-4655-b2a8-ae0bd8ebd3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:07.734532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b092c-1c6b-4343-a1e7-dc07c2a0271c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.739385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e5cac-f162-4053-a9db-eb632937c6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.740295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e5cac-f162-4053-a9db-eb632937c6a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.740612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e5cac-f162-4053-a9db-eb632937c6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.740849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e5cac-f162-4053-a9db-eb632937c6a9 admin admin] PATH_INFO: `/v3/projects/d08f2f4ac6d54f7484711b28f48aeeb3/users/a15a0abc2f25410b8476464d7cd32802/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.742186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 819/1610] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:07.743085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77317a-4621-4655-b2a8-ae0bd8ebd3bd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:07.753720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77317a-4621-4655-b2a8-ae0bd8ebd3bd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.758686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa33b4-75a4-49ac-a105-5161ded3dde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.759491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa33b4-75a4-49ac-a105-5161ded3dde8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.759839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa33b4-75a4-49ac-a105-5161ded3dde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.760184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa33b4-75a4-49ac-a105-5161ded3dde8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.760954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa33b4-75a4-49ac-a105-5161ded3dde8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.762459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e5cac-f162-4053-a9db-eb632937c6a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08f2f4ac6d54f7484711b28f48aeeb3, user_id=a15a0abc2f25410b8476464d7cd32802, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.763020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa33b4-75a4-49ac-a105-5161ded3dde8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.765335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e5cac-f162-4053-a9db-eb632937c6a9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.782072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 792/1611] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:07 2022] PUT /identity/v3/projects/d08f2f4ac6d54f7484711b28f48aeeb3/users/a15a0abc2f25410b8476464d7cd32802/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:07.783425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e5cac-f162-4053-a9db-eb632937c6a9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:07.792663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 820/1612] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:07.794106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39a10-8e11-456b-9401-57dcc2b15d36 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.794463 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39a10-8e11-456b-9401-57dcc2b15d36 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.794810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39a10-8e11-456b-9401-57dcc2b15d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.815303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aee39a10-8e11-456b-9401-57dcc2b15d36 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:07.816502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aee39a10-8e11-456b-9401-57dcc2b15d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:07.830937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:07.879533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e5cac-f162-4053-a9db-eb632937c6a9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.903359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 821/1613] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:07.905857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee39a10-8e11-456b-9401-57dcc2b15d36 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:07.918036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee39a10-8e11-456b-9401-57dcc2b15d36 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:07.922765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133e2e-7070-426e-b966-469baecc211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.923489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133e2e-7070-426e-b966-469baecc211d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.923773 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133e2e-7070-426e-b966-469baecc211d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.924101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133e2e-7070-426e-b966-469baecc211d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.924812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b133e2e-7070-426e-b966-469baecc211d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.927091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b133e2e-7070-426e-b966-469baecc211d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.941454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:07.968633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 822/1614] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:07.968633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42d7b-c256-4f2e-946a-bd552d66f444 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.968633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42d7b-c256-4f2e-946a-bd552d66f444 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.968633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42d7b-c256-4f2e-946a-bd552d66f444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.973881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7ab9-8ed5-46cb-a2f9-922127312d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:07.975007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7ab9-8ed5-46cb-a2f9-922127312d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:07.975378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7ab9-8ed5-46cb-a2f9-922127312d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:07.975650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7ab9-8ed5-46cb-a2f9-922127312d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:07.976516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af7ab9-8ed5-46cb-a2f9-922127312d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:07.978813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af7ab9-8ed5-46cb-a2f9-922127312d1b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:07.991131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-58c42d7b-c256-4f2e-946a-bd552d66f444 None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:07.992775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58c42d7b-c256-4f2e-946a-bd552d66f444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:08.017521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 793/1615] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/projects => generated 365 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:08.019555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7ab9-8ed5-46cb-a2f9-922127312d1b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.076196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 823/1616] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 23 08:20:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:08.078106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e897f8-8d26-4587-924c-5ab763652f8f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.078553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e897f8-8d26-4587-924c-5ab763652f8f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.078976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e897f8-8d26-4587-924c-5ab763652f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.100376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-22e897f8-8d26-4587-924c-5ab763652f8f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:08.103302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-22e897f8-8d26-4587-924c-5ab763652f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:08.106807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7ab9-8ed5-46cb-a2f9-922127312d1b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.112637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b55a9-de5d-4b3b-892c-5d899ed0ad31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.113715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b55a9-de5d-4b3b-892c-5d899ed0ad31 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.115071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b55a9-de5d-4b3b-892c-5d899ed0ad31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.115756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b55a9-de5d-4b3b-892c-5d899ed0ad31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.193170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 824/1617] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:08.194294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e897f8-8d26-4587-924c-5ab763652f8f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.213842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e897f8-8d26-4587-924c-5ab763652f8f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.219158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b55a9-de5d-4b3b-892c-5d899ed0ad31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.220467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b55a9-de5d-4b3b-892c-5d899ed0ad31 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.223939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9bb27-aa30-44a3-a051-8b1147d25e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.225130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9bb27-aa30-44a3-a051-8b1147d25e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.225581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9bb27-aa30-44a3-a051-8b1147d25e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.226269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9bb27-aa30-44a3-a051-8b1147d25e70 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.227805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9bb27-aa30-44a3-a051-8b1147d25e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.231551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9bb27-aa30-44a3-a051-8b1147d25e70 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.233766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 794/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:08.235403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b55a9-de5d-4b3b-892c-5d899ed0ad31 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.247051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b55a9-de5d-4b3b-892c-5d899ed0ad31 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.254704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc5a74-0aed-47e0-91c8-eac78aa2c6a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.255513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5a74-0aed-47e0-91c8-eac78aa2c6a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.255868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5a74-0aed-47e0-91c8-eac78aa2c6a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.256224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5a74-0aed-47e0-91c8-eac78aa2c6a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.264293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 825/1619] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:08 2022] POST /identity/v3/users => generated 297 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:08.265877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9bb27-aa30-44a3-a051-8b1147d25e70 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.270316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc5a74-0aed-47e0-91c8-eac78aa2c6a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.272027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc5a74-0aed-47e0-91c8-eac78aa2c6a0 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.276921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:08.281437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9bb27-aa30-44a3-a051-8b1147d25e70 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.284312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 795/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:08.285408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc5a74-0aed-47e0-91c8-eac78aa2c6a0 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.290356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2387a2-3043-4709-99f2-e3a2d4a7afbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.294391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2387a2-3043-4709-99f2-e3a2d4a7afbd admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.295864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2387a2-3043-4709-99f2-e3a2d4a7afbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.296763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2387a2-3043-4709-99f2-e3a2d4a7afbd admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.298188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2387a2-3043-4709-99f2-e3a2d4a7afbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.300303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc5a74-0aed-47e0-91c8-eac78aa2c6a0 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.301400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2387a2-3043-4709-99f2-e3a2d4a7afbd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.311040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e54c-2f9c-4c77-a15d-218d603c9054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.311890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e54c-2f9c-4c77-a15d-218d603c9054 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.312164 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e54c-2f9c-4c77-a15d-218d603c9054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.312629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e54c-2f9c-4c77-a15d-218d603c9054 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.313413 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033e54c-2f9c-4c77-a15d-218d603c9054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.315325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033e54c-2f9c-4c77-a15d-218d603c9054 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.328404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 826/1621] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:08 2022] POST /identity/v3/users => generated 292 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:08.329337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2387a2-3043-4709-99f2-e3a2d4a7afbd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.348231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 796/1622] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:08 2022] POST /identity/v3/users => generated 289 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:08.349273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e54c-2f9c-4c77-a15d-218d603c9054 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.365957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e54c-2f9c-4c77-a15d-218d603c9054 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.375302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e44f4-0b27-49c5-b50a-31c0eae5f1a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.376745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e44f4-0b27-49c5-b50a-31c0eae5f1a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.377344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e44f4-0b27-49c5-b50a-31c0eae5f1a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.377835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e44f4-0b27-49c5-b50a-31c0eae5f1a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.395767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e44f4-0b27-49c5-b50a-31c0eae5f1a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.398036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:08.398643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e44f4-0b27-49c5-b50a-31c0eae5f1a5 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.425480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 797/1623] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:08.426903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e44f4-0b27-49c5-b50a-31c0eae5f1a5 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.444066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e44f4-0b27-49c5-b50a-31c0eae5f1a5 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.452118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd8816-bbf3-46ac-a440-3fabfaca15b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.454010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd8816-bbf3-46ac-a440-3fabfaca15b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.454400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd8816-bbf3-46ac-a440-3fabfaca15b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.454797 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd8816-bbf3-46ac-a440-3fabfaca15b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.456729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd8816-bbf3-46ac-a440-3fabfaca15b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.458628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2387a2-3043-4709-99f2-e3a2d4a7afbd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.459233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd8816-bbf3-46ac-a440-3fabfaca15b7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.471991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49bf1f-b9c3-4b7f-b097-8fef061ee0d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.472701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49bf1f-b9c3-4b7f-b097-8fef061ee0d3 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.472998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49bf1f-b9c3-4b7f-b097-8fef061ee0d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.473272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49bf1f-b9c3-4b7f-b097-8fef061ee0d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.493801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 798/1624] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:08.494954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd8816-bbf3-46ac-a440-3fabfaca15b7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.511779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd8816-bbf3-46ac-a440-3fabfaca15b7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.531434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f929c2-d998-4623-a7d5-e921ee90a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.532175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f929c2-d998-4623-a7d5-e921ee90a992 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.532497 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f929c2-d998-4623-a7d5-e921ee90a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.532774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f929c2-d998-4623-a7d5-e921ee90a992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.533716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f929c2-d998-4623-a7d5-e921ee90a992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.536109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f929c2-d998-4623-a7d5-e921ee90a992 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.567383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 799/1625] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:08.568869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f929c2-d998-4623-a7d5-e921ee90a992 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.591852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f929c2-d998-4623-a7d5-e921ee90a992 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.599490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49bf1f-b9c3-4b7f-b097-8fef061ee0d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.601937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49bf1f-b9c3-4b7f-b097-8fef061ee0d3 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.605649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebfdc1-3588-4201-a8ed-c8f55a7639e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.606382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebfdc1-3588-4201-a8ed-c8f55a7639e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.606664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebfdc1-3588-4201-a8ed-c8f55a7639e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.606987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebfdc1-3588-4201-a8ed-c8f55a7639e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.607795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebfdc1-3588-4201-a8ed-c8f55a7639e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.612738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebfdc1-3588-4201-a8ed-c8f55a7639e1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.628005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 827/1626] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:08.630346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49bf1f-b9c3-4b7f-b097-8fef061ee0d3 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.637057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 800/1627] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:08.637824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebfdc1-3588-4201-a8ed-c8f55a7639e1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.652517 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49bf1f-b9c3-4b7f-b097-8fef061ee0d3 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.659991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebfdc1-3588-4201-a8ed-c8f55a7639e1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.664278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef72aa-cf3f-4331-b598-2968bf60d721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.665655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef72aa-cf3f-4331-b598-2968bf60d721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.666468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef72aa-cf3f-4331-b598-2968bf60d721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.666750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef72aa-cf3f-4331-b598-2968bf60d721 admin admin] PATH_INFO: `/v3/projects/2e796e1706e940d59eb7aa54bd48364a/users/c8746086da8947f68a212a3fdc520dae/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.670641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83503a4f-b6af-4232-b013-32fe45eea3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.673545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83503a4f-b6af-4232-b013-32fe45eea3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.673892 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83503a4f-b6af-4232-b013-32fe45eea3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.675061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83503a4f-b6af-4232-b013-32fe45eea3e9 admin admin] PATH_INFO: `/v3/projects/62fb6fc7865b49048e153e13d8ac359d/users/2eb9aeb920d1446b904e42b07b6bab4a/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.717678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef72aa-cf3f-4331-b598-2968bf60d721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e796e1706e940d59eb7aa54bd48364a, user_id=c8746086da8947f68a212a3fdc520dae, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.722089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83503a4f-b6af-4232-b013-32fe45eea3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62fb6fc7865b49048e153e13d8ac359d, user_id=2eb9aeb920d1446b904e42b07b6bab4a, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.724518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef72aa-cf3f-4331-b598-2968bf60d721 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.728047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83503a4f-b6af-4232-b013-32fe45eea3e9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.754308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 828/1628] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:08 2022] PUT /identity/v3/projects/2e796e1706e940d59eb7aa54bd48364a/users/c8746086da8947f68a212a3fdc520dae/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:08.755992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef72aa-cf3f-4331-b598-2968bf60d721 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.760082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 801/1629] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:08 2022] PUT /identity/v3/projects/62fb6fc7865b49048e153e13d8ac359d/users/2eb9aeb920d1446b904e42b07b6bab4a/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:08.762841 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83503a4f-b6af-4232-b013-32fe45eea3e9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.903178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef72aa-cf3f-4331-b598-2968bf60d721 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.904025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83503a4f-b6af-4232-b013-32fe45eea3e9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.912889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7fecd-ed94-4c44-a09e-457c93f02948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.913690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fadd5-4170-468a-a3be-59c91241e528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:08.915557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fadd5-4170-468a-a3be-59c91241e528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.916362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7fecd-ed94-4c44-a09e-457c93f02948 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:08.916362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7fecd-ed94-4c44-a09e-457c93f02948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.916362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7fecd-ed94-4c44-a09e-457c93f02948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.916889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fadd5-4170-468a-a3be-59c91241e528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:08.917700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7fecd-ed94-4c44-a09e-457c93f02948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.918077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fadd5-4170-468a-a3be-59c91241e528 admin admin] PATH_INFO: `/v3/projects/3f7b3690abc74b5797251b8ae2968e16/users/57b74d81419e4c78a0a687a58d0e96b0/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:08.921084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7fecd-ed94-4c44-a09e-457c93f02948 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.950476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 802/1630] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:08.952826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7fecd-ed94-4c44-a09e-457c93f02948 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:08.956212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fadd5-4170-468a-a3be-59c91241e528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7b3690abc74b5797251b8ae2968e16, user_id=57b74d81419e4c78a0a687a58d0e96b0, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:08.960748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fadd5-4170-468a-a3be-59c91241e528 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:08.978748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7fecd-ed94-4c44-a09e-457c93f02948 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:08.987742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 829/1631] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:08 2022] PUT /identity/v3/projects/3f7b3690abc74b5797251b8ae2968e16/users/57b74d81419e4c78a0a687a58d0e96b0/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:08.994366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fadd5-4170-468a-a3be-59c91241e528 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:09.109750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6938832-3e91-4fb2-b5d1-94bf96847b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:09.110688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6938832-3e91-4fb2-b5d1-94bf96847b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.113411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6938832-3e91-4fb2-b5d1-94bf96847b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.113780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6938832-3e91-4fb2-b5d1-94bf96847b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.114645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6938832-3e91-4fb2-b5d1-94bf96847b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:09.116203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:09.118672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6938832-3e91-4fb2-b5d1-94bf96847b0d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:09.143717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 803/1632] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:09.145757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6938832-3e91-4fb2-b5d1-94bf96847b0d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:09.157775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:09.159629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fadd5-4170-468a-a3be-59c91241e528 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:09.168125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6938832-3e91-4fb2-b5d1-94bf96847b0d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:09.172516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58009878-bc7b-436d-a1fd-e1a7f17f4db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:09.173592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58009878-bc7b-436d-a1fd-e1a7f17f4db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.173982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58009878-bc7b-436d-a1fd-e1a7f17f4db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.174359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58009878-bc7b-436d-a1fd-e1a7f17f4db4 admin admin] PATH_INFO: `/v3/projects/2e796e1706e940d59eb7aa54bd48364a/users/c8746086da8947f68a212a3fdc520dae/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.179782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d12b4-12f9-4346-a85e-6afd59618c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:09.180485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d12b4-12f9-4346-a85e-6afd59618c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.180768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d12b4-12f9-4346-a85e-6afd59618c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.181052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d12b4-12f9-4346-a85e-6afd59618c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.181715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58009878-bc7b-436d-a1fd-e1a7f17f4db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e796e1706e940d59eb7aa54bd48364a, user_id=c8746086da8947f68a212a3fdc520dae, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:09.183970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d12b4-12f9-4346-a85e-6afd59618c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:09.184806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58009878-bc7b-436d-a1fd-e1a7f17f4db4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:09.190916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d12b4-12f9-4346-a85e-6afd59618c50 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:09.224558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 804/1634] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:09.225525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d12b4-12f9-4346-a85e-6afd59618c50 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:09.230947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 830/1634] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:08 2022] PUT /identity/v3/projects/2e796e1706e940d59eb7aa54bd48364a/users/c8746086da8947f68a212a3fdc520dae/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:09.232265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a997-8411-4dce-aa05-66bab7403df4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.233565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a997-8411-4dce-aa05-66bab7403df4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.233970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a997-8411-4dce-aa05-66bab7403df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.261457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d12b4-12f9-4346-a85e-6afd59618c50 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:09.270128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8221a997-8411-4dce-aa05-66bab7403df4 None None] MFA Rules not processed for user `c8746086da8947f68a212a3fdc520dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:09.271376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8221a997-8411-4dce-aa05-66bab7403df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:09.376719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 831/1635] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:09.377830 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221a997-8411-4dce-aa05-66bab7403df4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:09.449633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221a997-8411-4dce-aa05-66bab7403df4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:09.463589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59e791-1e97-42e3-84e4-324a1db9e013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:09.464512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e791-1e97-42e3-84e4-324a1db9e013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.464800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e791-1e97-42e3-84e4-324a1db9e013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.465073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e791-1e97-42e3-84e4-324a1db9e013 admin admin] PATH_INFO: `/v3/projects/62fb6fc7865b49048e153e13d8ac359d/users/2eb9aeb920d1446b904e42b07b6bab4a/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.467300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4527e7e-0d95-44bd-bf9c-a7e6515c2b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:09.471069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59e791-1e97-42e3-84e4-324a1db9e013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62fb6fc7865b49048e153e13d8ac359d, user_id=2eb9aeb920d1446b904e42b07b6bab4a, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:09.472078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4527e7e-0d95-44bd-bf9c-a7e6515c2b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.472464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4527e7e-0d95-44bd-bf9c-a7e6515c2b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.472841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4527e7e-0d95-44bd-bf9c-a7e6515c2b23 admin admin] PATH_INFO: `/v3/projects/3f7b3690abc74b5797251b8ae2968e16/users/57b74d81419e4c78a0a687a58d0e96b0/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.475985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4527e7e-0d95-44bd-bf9c-a7e6515c2b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7b3690abc74b5797251b8ae2968e16, user_id=57b74d81419e4c78a0a687a58d0e96b0, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:09.478918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59e791-1e97-42e3-84e4-324a1db9e013 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:09.481226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4527e7e-0d95-44bd-bf9c-a7e6515c2b23 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:09.520769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 805/1636] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:09 2022] PUT /identity/v3/projects/62fb6fc7865b49048e153e13d8ac359d/users/2eb9aeb920d1446b904e42b07b6bab4a/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:09.520769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb41d-bb44-4249-87a1-8722fc349c2f None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.520769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb41d-bb44-4249-87a1-8722fc349c2f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.520769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb41d-bb44-4249-87a1-8722fc349c2f None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.520769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 806/1637] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 23 08:20:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:09.524083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cb41d-bb44-4249-87a1-8722fc349c2f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:09.540861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 832/1638] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:09 2022] PUT /identity/v3/projects/3f7b3690abc74b5797251b8ae2968e16/users/57b74d81419e4c78a0a687a58d0e96b0/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:09.544442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba9907-a451-4c8d-8b8d-3629b0136ce8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.545900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba9907-a451-4c8d-8b8d-3629b0136ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.547102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba9907-a451-4c8d-8b8d-3629b0136ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.623518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-20ba9907-a451-4c8d-8b8d-3629b0136ce8 None None] MFA Rules not processed for user `2eb9aeb920d1446b904e42b07b6bab4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:09.627294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-20ba9907-a451-4c8d-8b8d-3629b0136ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:09.679662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cb41d-bb44-4249-87a1-8722fc349c2f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:09.683991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4542154-73b1-4d93-bfd9-6bc953899639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:09.684843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4542154-73b1-4d93-bfd9-6bc953899639 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.685256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4542154-73b1-4d93-bfd9-6bc953899639 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.685535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4542154-73b1-4d93-bfd9-6bc953899639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.739342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:09.775607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 833/1639] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:09.777085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c3f8b-d16d-4873-8202-730c94631ea4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.777591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c3f8b-d16d-4873-8202-730c94631ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.778342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c3f8b-d16d-4873-8202-730c94631ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.801073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0f9c3f8b-d16d-4873-8202-730c94631ea4 None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:09.802599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0f9c3f8b-d16d-4873-8202-730c94631ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:09.834178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4542154-73b1-4d93-bfd9-6bc953899639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:09.838596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4542154-73b1-4d93-bfd9-6bc953899639 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:09.858318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 807/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:09.860711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee37be-0768-4fb8-9a6f-31fa1142e02f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.862152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee37be-0768-4fb8-9a6f-31fa1142e02f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.865530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee37be-0768-4fb8-9a6f-31fa1142e02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.934099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 834/1641] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:20:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:09.937680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c3f8b-d16d-4873-8202-730c94631ea4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:09.952295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b6ee37be-0768-4fb8-9a6f-31fa1142e02f None None] MFA Rules not processed for user `57b74d81419e4c78a0a687a58d0e96b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:09.960563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b6ee37be-0768-4fb8-9a6f-31fa1142e02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:09.961777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c3f8b-d16d-4873-8202-730c94631ea4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:09.973247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d871fc0-25b0-4956-8111-381ebe08783b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:09.974195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d871fc0-25b0-4956-8111-381ebe08783b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:09.974548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d871fc0-25b0-4956-8111-381ebe08783b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:09.974830 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d871fc0-25b0-4956-8111-381ebe08783b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:09.992175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d871fc0-25b0-4956-8111-381ebe08783b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:09.996451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d871fc0-25b0-4956-8111-381ebe08783b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:10.021993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 835/1642] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:10.031564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:10.040263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d871fc0-25b0-4956-8111-381ebe08783b service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:10.064281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d871fc0-25b0-4956-8111-381ebe08783b service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:10.076260 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6fbd6-a94e-45cd-89bc-1d0593782dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:10.076600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6fbd6-a94e-45cd-89bc-1d0593782dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:10.077661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6fbd6-a94e-45cd-89bc-1d0593782dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:10.078018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6fbd6-a94e-45cd-89bc-1d0593782dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:10.109481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6fbd6-a94e-45cd-89bc-1d0593782dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:10.121908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6fbd6-a94e-45cd-89bc-1d0593782dd5 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:10.151655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 808/1643] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:10.159805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 836/1644] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:10.162716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee37be-0768-4fb8-9a6f-31fa1142e02f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:10.180619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee37be-0768-4fb8-9a6f-31fa1142e02f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:10.188055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28517181-e9da-4942-ba5e-b347d9406762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:10.188627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517181-e9da-4942-ba5e-b347d9406762 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:10.188906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517181-e9da-4942-ba5e-b347d9406762 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:10.189182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517181-e9da-4942-ba5e-b347d9406762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:10.285250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28517181-e9da-4942-ba5e-b347d9406762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:10.287480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28517181-e9da-4942-ba5e-b347d9406762 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:10.320931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 809/1645] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:10.715820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6fbd6-a94e-45cd-89bc-1d0593782dd5 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:10.729736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6fbd6-a94e-45cd-89bc-1d0593782dd5 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:10.739975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73baf44-e99f-4246-b556-b5664af2922c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:10.742947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73baf44-e99f-4246-b556-b5664af2922c service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:10.743251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73baf44-e99f-4246-b556-b5664af2922c service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:10.752366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73baf44-e99f-4246-b556-b5664af2922c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:10.781356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73baf44-e99f-4246-b556-b5664af2922c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:10.784075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73baf44-e99f-4246-b556-b5664af2922c service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:10.808506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 837/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:11.298329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28517181-e9da-4942-ba5e-b347d9406762 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:11.308227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28517181-e9da-4942-ba5e-b347d9406762 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:11.344746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d1762-73c3-4500-90fc-1aac991d90b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:11.345433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d1762-73c3-4500-90fc-1aac991d90b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:11.345713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d1762-73c3-4500-90fc-1aac991d90b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:11.346048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d1762-73c3-4500-90fc-1aac991d90b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:11.505390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d1762-73c3-4500-90fc-1aac991d90b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:11.506863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d1762-73c3-4500-90fc-1aac991d90b5 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:11.523013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 810/1647] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:11 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:11.929179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1946c-cd22-423f-bc9d-2465efdf19c2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:11.930671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1946c-cd22-423f-bc9d-2465efdf19c2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:11.931385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1946c-cd22-423f-bc9d-2465efdf19c2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:11.942653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 838/1648] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 23 08:20:11 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:11.953039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff672-5583-43a2-a234-c03086e562bb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:11.953396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff672-5583-43a2-a234-c03086e562bb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:11.953733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff672-5583-43a2-a234-c03086e562bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:12.034395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ba8ff672-5583-43a2-a234-c03086e562bb None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:12.035669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ba8ff672-5583-43a2-a234-c03086e562bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:12.082146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 811/1649] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 23 08:20:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:12.086378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29578b2b-cada-4c54-9dd9-c4b27fde336d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:12.086668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29578b2b-cada-4c54-9dd9-c4b27fde336d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:12.086950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29578b2b-cada-4c54-9dd9-c4b27fde336d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:12.088413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 839/1650] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 23 08:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:12.094883 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2c084-8d7c-4d58-b79c-0681e6ecc1ed None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:12.095444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2c084-8d7c-4d58-b79c-0681e6ecc1ed None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:12.096024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2c084-8d7c-4d58-b79c-0681e6ecc1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:12.170704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-02e2c084-8d7c-4d58-b79c-0681e6ecc1ed None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:12.181485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-02e2c084-8d7c-4d58-b79c-0681e6ecc1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:12.213088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 812/1651] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:20:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:12.315166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29578b2b-cada-4c54-9dd9-c4b27fde336d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:12.334898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29578b2b-cada-4c54-9dd9-c4b27fde336d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:12.353488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b16de1-d7bb-4c3d-b343-3d296144e5ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:12.354778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16de1-d7bb-4c3d-b343-3d296144e5ff service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:12.355114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16de1-d7bb-4c3d-b343-3d296144e5ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:12.355423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16de1-d7bb-4c3d-b343-3d296144e5ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:12.395731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b16de1-d7bb-4c3d-b343-3d296144e5ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:12.399458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b16de1-d7bb-4c3d-b343-3d296144e5ff service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:12.433385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 840/1652] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:12.810689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2c084-8d7c-4d58-b79c-0681e6ecc1ed None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:12.868126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:12.931690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2c084-8d7c-4d58-b79c-0681e6ecc1ed None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:12.962037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:12.965974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:12.967384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:12.967827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] PATH_INFO: `/v3/users/5d4550977e4b47738deb7747899ef5e6` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:12.970128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4550977e4b47738deb7747899ef5e6)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:12.973408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:20:12.973408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jun 23 08:20:12.973408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: Jun 23 08:20:12.977991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:13.106425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4550977e4b47738deb7747899ef5e6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:13.126653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4550977e4b47738deb7747899ef5e6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:13.158749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4550977e4b47738deb7747899ef5e6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:13.159610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4550977e4b47738deb7747899ef5e6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:13.200491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4550977e4b47738deb7747899ef5e6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:13.212121 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 813/1653] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:12 2022] DELETE /identity/v3/users/5d4550977e4b47738deb7747899ef5e6 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:13.217462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce43961-1f53-4654-bd6d-53080036330f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:13.219197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce43961-1f53-4654-bd6d-53080036330f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:13.220191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce43961-1f53-4654-bd6d-53080036330f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:13.256170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6ce43961-1f53-4654-bd6d-53080036330f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:13.262850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6ce43961-1f53-4654-bd6d-53080036330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:13.466546 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 841/1654] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:13.523312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:13.525683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:13.749988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172e35e-0a17-43fd-9fb6-5112ac6da4dd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:13.768276 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a34f1-57fb-46d7-b225-6d3971ce5780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:13.768977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a34f1-57fb-46d7-b225-6d3971ce5780 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:13.776921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a34f1-57fb-46d7-b225-6d3971ce5780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:13.777292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a34f1-57fb-46d7-b225-6d3971ce5780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:13.819727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a34f1-57fb-46d7-b225-6d3971ce5780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:13.821268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a34f1-57fb-46d7-b225-6d3971ce5780 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:13.852737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 814/1655] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:15.643808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce43961-1f53-4654-bd6d-53080036330f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:15.646334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6ce43961-1f53-4654-bd6d-53080036330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:15.685985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce43961-1f53-4654-bd6d-53080036330f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:15.714355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:15.717021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:15.718171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:15.720540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] PATH_INFO: `/v3/projects/e5c2f087908e492f8874200a56048571` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:15.726610 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c2f087908e492f8874200a56048571)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:15.729396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 08:20:15.729396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jun 23 08:20:15.729396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: Jun 23 08:20:15.737114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:15.824709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:15.938820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c2f087908e492f8874200a56048571', 'request_id': 'req-3cd74ab0-411a-48ec-ad11-44591d237aa1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:15.961795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c2f087908e492f8874200a56048571', 'request_id': 'req-3cd74ab0-411a-48ec-ad11-44591d237aa1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:15.975819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c2f087908e492f8874200a56048571', 'request_id': 'req-3cd74ab0-411a-48ec-ad11-44591d237aa1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:16.021001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] The token cache is being invalidate because project e5c2f087908e492f8874200a56048571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:16.024533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:16.025550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3cd74ab0-411a-48ec-ad11-44591d237aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:16.030692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 842/1656] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:15 2022] DELETE /identity/v3/projects/e5c2f087908e492f8874200a56048571 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:16.031959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a34f1-57fb-46d7-b225-6d3971ce5780 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:16.055161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a34f1-57fb-46d7-b225-6d3971ce5780 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:16.284840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:16.292783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:16.300743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:16.301310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] PATH_INFO: `/v3/users/2eb9aeb920d1446b904e42b07b6bab4a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:16.303663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb9aeb920d1446b904e42b07b6bab4a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:16.313754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:16.335201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:16.456870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb9aeb920d1446b904e42b07b6bab4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:16.475800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb9aeb920d1446b904e42b07b6bab4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:16.507484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb9aeb920d1446b904e42b07b6bab4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:16.508510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb9aeb920d1446b904e42b07b6bab4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:16.539147 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb9aeb920d1446b904e42b07b6bab4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:16.572035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 815/1657] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:16 2022] DELETE /identity/v3/users/2eb9aeb920d1446b904e42b07b6bab4a => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:16.934706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd637b-321e-45d4-9781-ff5c210740ad None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:16.935303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd637b-321e-45d4-9781-ff5c210740ad None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:16.935467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd637b-321e-45d4-9781-ff5c210740ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:17.040582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a3dd637b-321e-45d4-9781-ff5c210740ad None None] MFA Rules not processed for user `9b178568bc5e4876bdab388ac3302f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:17.042170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a3dd637b-321e-45d4-9781-ff5c210740ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:17.215550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 843/1658] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:17.233993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:17.236557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:17.437891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5b2e2-e43f-49b7-8d53-d6d51bfee2e2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:17.452965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198decf-8b1f-4c35-9801-4545dfc58388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:17.455064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198decf-8b1f-4c35-9801-4545dfc58388 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:17.455613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198decf-8b1f-4c35-9801-4545dfc58388 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:17.456180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198decf-8b1f-4c35-9801-4545dfc58388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:17.511274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198decf-8b1f-4c35-9801-4545dfc58388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:17.520092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198decf-8b1f-4c35-9801-4545dfc58388 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:17.567046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 816/1659] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:18.013015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd637b-321e-45d4-9781-ff5c210740ad None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:18.014658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a3dd637b-321e-45d4-9781-ff5c210740ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:18.192750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd637b-321e-45d4-9781-ff5c210740ad None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:18.213049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:18.214143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:18.214800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:18.215313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] PATH_INFO: `/v3/projects/62fb6fc7865b49048e153e13d8ac359d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:18.217096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62fb6fc7865b49048e153e13d8ac359d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:18.219530 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:18.309795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:18.326104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62fb6fc7865b49048e153e13d8ac359d', 'request_id': 'req-5474a56e-791e-4432-96b1-ea90f1ad12c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:18.334186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62fb6fc7865b49048e153e13d8ac359d', 'request_id': 'req-5474a56e-791e-4432-96b1-ea90f1ad12c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:18.350135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62fb6fc7865b49048e153e13d8ac359d', 'request_id': 'req-5474a56e-791e-4432-96b1-ea90f1ad12c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:18.382150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] The token cache is being invalidate because project 62fb6fc7865b49048e153e13d8ac359d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:18.382368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:18.388653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5474a56e-791e-4432-96b1-ea90f1ad12c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:18.390631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 844/1660] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:18 2022] DELETE /identity/v3/projects/62fb6fc7865b49048e153e13d8ac359d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:18.395869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805155bb-f140-4f5f-aa13-1784d182cd9c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:18.396937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805155bb-f140-4f5f-aa13-1784d182cd9c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:18.397460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805155bb-f140-4f5f-aa13-1784d182cd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:18.424041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-805155bb-f140-4f5f-aa13-1784d182cd9c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:18.426331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-805155bb-f140-4f5f-aa13-1784d182cd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:18.475053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1d351-e7d9-4e9a-9b38-a79ba761404f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:18.475140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1d351-e7d9-4e9a-9b38-a79ba761404f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:18.475851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1d351-e7d9-4e9a-9b38-a79ba761404f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:18.556463 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9ca1d351-e7d9-4e9a-9b38-a79ba761404f None None] MFA Rules not processed for user `a15a0abc2f25410b8476464d7cd32802`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:18.560741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9ca1d351-e7d9-4e9a-9b38-a79ba761404f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:18.625199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 817/1661] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:18.630699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d76cb7-f88a-4a81-a177-47123de5ef35 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:18.631182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d76cb7-f88a-4a81-a177-47123de5ef35 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:18.631638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d76cb7-f88a-4a81-a177-47123de5ef35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:18.678600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-49d76cb7-f88a-4a81-a177-47123de5ef35 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:18.680939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-49d76cb7-f88a-4a81-a177-47123de5ef35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:18.692043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 818/1662] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:18.697527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d76cb7-f88a-4a81-a177-47123de5ef35 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:18.709063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:18.737607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d76cb7-f88a-4a81-a177-47123de5ef35 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:18.755243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 845/1663] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:18.761641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2731-972e-4e75-86bc-338f5d05e062 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:18.761936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2731-972e-4e75-86bc-338f5d05e062 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:18.762220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2731-972e-4e75-86bc-338f5d05e062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:18.764548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:18.766300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:18.767288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:18.768691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:18.769569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:18.770669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:18.778899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:18.817104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e06c2731-972e-4e75-86bc-338f5d05e062 None None] MFA Rules not processed for user `a15a0abc2f25410b8476464d7cd32802`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:18.818625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e06c2731-972e-4e75-86bc-338f5d05e062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:18.821758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 819/1664] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:18.828784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:18.843488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 846/1665] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:18.849065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412c9a-c242-4aeb-9310-f3fca616a516 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:18.853215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c2731-972e-4e75-86bc-338f5d05e062 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:18.854716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e06c2731-972e-4e75-86bc-338f5d05e062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:18.855448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dce306-69ed-452b-a048-537a9e1f75d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:18.856394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dce306-69ed-452b-a048-537a9e1f75d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:18.856821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dce306-69ed-452b-a048-537a9e1f75d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:18.857237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dce306-69ed-452b-a048-537a9e1f75d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:18.858134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dce306-69ed-452b-a048-537a9e1f75d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:18.860467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dce306-69ed-452b-a048-537a9e1f75d9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:18.899155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 820/1666] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:18 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:18.905136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dce306-69ed-452b-a048-537a9e1f75d9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:19.106022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c2731-972e-4e75-86bc-338f5d05e062 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:19.112997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dce306-69ed-452b-a048-537a9e1f75d9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:19.125876 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4b34-cc3d-4472-9cbc-5187471da4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:19.127175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4b34-cc3d-4472-9cbc-5187471da4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:19.127818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4b34-cc3d-4472-9cbc-5187471da4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:19.128291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4b34-cc3d-4472-9cbc-5187471da4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:19.129208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4b34-cc3d-4472-9cbc-5187471da4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:19.131471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4b34-cc3d-4472-9cbc-5187471da4a4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:19.169611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 821/1667] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:18 2022] POST /identity/v3/users => generated 282 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:19.176366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4b34-cc3d-4472-9cbc-5187471da4a4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:19.199814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4b34-cc3d-4472-9cbc-5187471da4a4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:19.217162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606500d7-982a-42ce-850a-8426096d36f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:19.218438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606500d7-982a-42ce-850a-8426096d36f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:19.219099 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606500d7-982a-42ce-850a-8426096d36f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:19.219620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606500d7-982a-42ce-850a-8426096d36f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:19.220566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606500d7-982a-42ce-850a-8426096d36f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:19.222426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606500d7-982a-42ce-850a-8426096d36f9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:19.257223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:19.262584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 822/1668] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:19.266320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606500d7-982a-42ce-850a-8426096d36f9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:19.299898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606500d7-982a-42ce-850a-8426096d36f9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:19.313290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce926d1-f62a-490c-bd68-3643bc809273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:19.316067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce926d1-f62a-490c-bd68-3643bc809273 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:19.316700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce926d1-f62a-490c-bd68-3643bc809273 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:19.318028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce926d1-f62a-490c-bd68-3643bc809273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:19.319955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795ba15-5d2d-4d2f-80de-0fccdbb58b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:19.321177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795ba15-5d2d-4d2f-80de-0fccdbb58b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:19.322971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795ba15-5d2d-4d2f-80de-0fccdbb58b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:19.323708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795ba15-5d2d-4d2f-80de-0fccdbb58b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:19.325249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795ba15-5d2d-4d2f-80de-0fccdbb58b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:19.329510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795ba15-5d2d-4d2f-80de-0fccdbb58b7a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:19.368198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 823/1669] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:19.371598 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795ba15-5d2d-4d2f-80de-0fccdbb58b7a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:19.401332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795ba15-5d2d-4d2f-80de-0fccdbb58b7a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:19.436645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f48f-219c-4223-b356-98065c124b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:19.443657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f48f-219c-4223-b356-98065c124b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:19.444048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f48f-219c-4223-b356-98065c124b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:19.444423 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f48f-219c-4223-b356-98065c124b1e admin admin] PATH_INFO: `/v3/projects/084a5b7c3611482a9fa62180ae3f2e9a/users/d76a04278e0b43738afcddbb13346d8c/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:19.504493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0f48f-219c-4223-b356-98065c124b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=084a5b7c3611482a9fa62180ae3f2e9a, user_id=d76a04278e0b43738afcddbb13346d8c, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:19.507582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0f48f-219c-4223-b356-98065c124b1e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:19.508876 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce926d1-f62a-490c-bd68-3643bc809273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:19.510559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce926d1-f62a-490c-bd68-3643bc809273 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:19.540686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 847/1670] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:20:19.567028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 824/1671] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:19 2022] PUT /identity/v3/projects/084a5b7c3611482a9fa62180ae3f2e9a/users/d76a04278e0b43738afcddbb13346d8c/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:19.583908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff457-44de-4c38-9d03-1cd2ce70391c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:19.584203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff457-44de-4c38-9d03-1cd2ce70391c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:19.584479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ff457-44de-4c38-9d03-1cd2ce70391c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:19.689603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6c9ff457-44de-4c38-9d03-1cd2ce70391c None None] MFA Rules not processed for user `d76a04278e0b43738afcddbb13346d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:19.691099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6c9ff457-44de-4c38-9d03-1cd2ce70391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:19.900134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 848/1672] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:19 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:19.905700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f48f-219c-4223-b356-98065c124b1e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:20.119638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f48f-219c-4223-b356-98065c124b1e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:20.145681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5576-7d10-4c18-a9e4-e88d462ab59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:20.146552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5576-7d10-4c18-a9e4-e88d462ab59d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:20.148075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5576-7d10-4c18-a9e4-e88d462ab59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:20.148544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5576-7d10-4c18-a9e4-e88d462ab59d admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:20.149435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff5576-7d10-4c18-a9e4-e88d462ab59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:20.151860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff5576-7d10-4c18-a9e4-e88d462ab59d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:20.221895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:20.250970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 825/1673] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:19 2022] POST /identity/v3/projects => generated 365 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:20.251723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ff457-44de-4c38-9d03-1cd2ce70391c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:20.277473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ff457-44de-4c38-9d03-1cd2ce70391c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:20.427047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e45822-6d5e-4065-b996-e5a307014870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:20.428074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e45822-6d5e-4065-b996-e5a307014870 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:20.430490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e45822-6d5e-4065-b996-e5a307014870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:20.430948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e45822-6d5e-4065-b996-e5a307014870 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:20.432937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e45822-6d5e-4065-b996-e5a307014870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:20.438467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e45822-6d5e-4065-b996-e5a307014870 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:20.486446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 849/1674] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:20 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:20.488513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5576-7d10-4c18-a9e4-e88d462ab59d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:20.505949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5576-7d10-4c18-a9e4-e88d462ab59d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:20.519783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cc360-f6df-47ff-87c9-4fcf2cf6aa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:20.520513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc360-f6df-47ff-87c9-4fcf2cf6aa40 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:20.520785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc360-f6df-47ff-87c9-4fcf2cf6aa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:20.521279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc360-f6df-47ff-87c9-4fcf2cf6aa40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:20.522386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cc360-f6df-47ff-87c9-4fcf2cf6aa40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:20.524239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cc360-f6df-47ff-87c9-4fcf2cf6aa40 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:20.565745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 826/1675] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:20.571746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e45822-6d5e-4065-b996-e5a307014870 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:20.579772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:20.587143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e45822-6d5e-4065-b996-e5a307014870 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:20.600267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c21a8d-dafe-4df8-b53b-b3e3554cb449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:20.601218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c21a8d-dafe-4df8-b53b-b3e3554cb449 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:20.601793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c21a8d-dafe-4df8-b53b-b3e3554cb449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:20.602588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c21a8d-dafe-4df8-b53b-b3e3554cb449 admin admin] PATH_INFO: `/v3/projects/b31cfb8695144d45bb10b67a69d5c866/users/324bd79b91e647acac91e20f146b358f/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:20.660475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c21a8d-dafe-4df8-b53b-b3e3554cb449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31cfb8695144d45bb10b67a69d5c866, user_id=324bd79b91e647acac91e20f146b358f, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:20.670933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c21a8d-dafe-4df8-b53b-b3e3554cb449 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:20.701145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cc360-f6df-47ff-87c9-4fcf2cf6aa40 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:20.701755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 850/1676] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:20 2022] PUT /identity/v3/projects/b31cfb8695144d45bb10b67a69d5c866/users/324bd79b91e647acac91e20f146b358f/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:20.726212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cc360-f6df-47ff-87c9-4fcf2cf6aa40 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:20.923986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e382f3-eb87-4efc-a775-8e19949085ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:20.924688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e382f3-eb87-4efc-a775-8e19949085ce admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:20.924967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e382f3-eb87-4efc-a775-8e19949085ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:20.937210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dad2b-f24f-407c-8b99-cf970b733e84 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:20.937494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dad2b-f24f-407c-8b99-cf970b733e84 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:20.937785 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dad2b-f24f-407c-8b99-cf970b733e84 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:20.939352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 851/1677] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 23 08:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:20.943725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e382f3-eb87-4efc-a775-8e19949085ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:20.948060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e382f3-eb87-4efc-a775-8e19949085ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:20.953922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ee320-e76c-47d5-865a-248b25b038b9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:20.954208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ee320-e76c-47d5-865a-248b25b038b9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:20.954509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ee320-e76c-47d5-865a-248b25b038b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:20.958941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e382f3-eb87-4efc-a775-8e19949085ce admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:21.002215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 827/1678] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:21.002906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e382f3-eb87-4efc-a775-8e19949085ce admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:21.053339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e382f3-eb87-4efc-a775-8e19949085ce admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:21.068873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fde60e-57f3-4c01-b1a8-d41a40bab67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:21.070160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fde60e-57f3-4c01-b1a8-d41a40bab67b admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:21.070575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fde60e-57f3-4c01-b1a8-d41a40bab67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:21.070846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fde60e-57f3-4c01-b1a8-d41a40bab67b admin admin] PATH_INFO: `/v3/projects/b31cfb8695144d45bb10b67a69d5c866/users/324bd79b91e647acac91e20f146b358f/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:21.075257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fde60e-57f3-4c01-b1a8-d41a40bab67b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31cfb8695144d45bb10b67a69d5c866, user_id=324bd79b91e647acac91e20f146b358f, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:21.076502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ad0ee320-e76c-47d5-865a-248b25b038b9 None None] MFA Rules not processed for user `2965b1c8ab52457f8f167acda4ba6844`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:21.080130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ad0ee320-e76c-47d5-865a-248b25b038b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:21.081740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fde60e-57f3-4c01-b1a8-d41a40bab67b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:21.114943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 828/1679] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:21 2022] PUT /identity/v3/projects/b31cfb8695144d45bb10b67a69d5c866/users/324bd79b91e647acac91e20f146b358f/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:21.116775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53711d4-6a70-4fc0-ac7d-d9a7d2c7a380 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:21.118238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53711d4-6a70-4fc0-ac7d-d9a7d2c7a380 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:21.118546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53711d4-6a70-4fc0-ac7d-d9a7d2c7a380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:21.216704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a53711d4-6a70-4fc0-ac7d-d9a7d2c7a380 None None] MFA Rules not processed for user `324bd79b91e647acac91e20f146b358f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:21.235034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a53711d4-6a70-4fc0-ac7d-d9a7d2c7a380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:21.279422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 852/1680] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:20:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:21.350557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ee320-e76c-47d5-865a-248b25b038b9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:21.352683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ad0ee320-e76c-47d5-865a-248b25b038b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:21.375925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ee320-e76c-47d5-865a-248b25b038b9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:21.519054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 829/1681] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:21.532465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53711d4-6a70-4fc0-ac7d-d9a7d2c7a380 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:21.535773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a53711d4-6a70-4fc0-ac7d-d9a7d2c7a380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:21.569784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2667c-0bfc-43d6-8c5a-dc9e31d0262b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:21.570461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2667c-0bfc-43d6-8c5a-dc9e31d0262b service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:21.572646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2667c-0bfc-43d6-8c5a-dc9e31d0262b service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:21.572943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2667c-0bfc-43d6-8c5a-dc9e31d0262b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:21.597330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2667c-0bfc-43d6-8c5a-dc9e31d0262b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:21.598834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2667c-0bfc-43d6-8c5a-dc9e31d0262b service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:21.646160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 853/1682] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:20:21.720411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53711d4-6a70-4fc0-ac7d-d9a7d2c7a380 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:21.737078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65286533-1ec9-4a5c-8ba6-7d96ffcce12e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:21.745714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286533-1ec9-4a5c-8ba6-7d96ffcce12e service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:21.745999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286533-1ec9-4a5c-8ba6-7d96ffcce12e service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:21.746278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286533-1ec9-4a5c-8ba6-7d96ffcce12e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:21.772161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65286533-1ec9-4a5c-8ba6-7d96ffcce12e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:21.773632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65286533-1ec9-4a5c-8ba6-7d96ffcce12e service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:21.778972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:21.803680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 830/1683] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:22.560056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d0707-179b-4175-a1a7-24fa84aa4aa4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:22.564229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d0707-179b-4175-a1a7-24fa84aa4aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:22.564996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d0707-179b-4175-a1a7-24fa84aa4aa4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:22.579299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 854/1684] 10.222.0.33 () {54 vars in 960 bytes} [Thu Jun 23 08:20:22 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:22.583751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc0c80-bc73-4da9-aabb-ae868092be55 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:22.584831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc0c80-bc73-4da9-aabb-ae868092be55 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:22.585307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc0c80-bc73-4da9-aabb-ae868092be55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:22.657965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5cdc0c80-bc73-4da9-aabb-ae868092be55 None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:22.665333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5cdc0c80-bc73-4da9-aabb-ae868092be55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:22.776489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9ccc4-0072-4a87-b43f-5e009218f3cf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:22.776901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9ccc4-0072-4a87-b43f-5e009218f3cf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:22.777423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9ccc4-0072-4a87-b43f-5e009218f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:22.840864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-88c9ccc4-0072-4a87-b43f-5e009218f3cf None None] MFA Rules not processed for user `a15a0abc2f25410b8476464d7cd32802`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:22.845536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-88c9ccc4-0072-4a87-b43f-5e009218f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:22.983842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 831/1685] 10.222.0.33 () {58 vars in 1057 bytes} [Thu Jun 23 08:20:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:22.993191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc0c80-bc73-4da9-aabb-ae868092be55 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:23.018454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc0c80-bc73-4da9-aabb-ae868092be55 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:23.039934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 855/1686] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:23.041809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9ccc4-0072-4a87-b43f-5e009218f3cf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:23.046660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1a4aa-3aea-4286-bd12-43053bb00cc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:23.050545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a4aa-3aea-4286-bd12-43053bb00cc3 service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:23.051336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a4aa-3aea-4286-bd12-43053bb00cc3 service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:23.052323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a4aa-3aea-4286-bd12-43053bb00cc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:23.058648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a2f1a4aa-3aea-4286-bd12-43053bb00cc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:23.095177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1a4aa-3aea-4286-bd12-43053bb00cc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:23.099095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1a4aa-3aea-4286-bd12-43053bb00cc3 service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:23.119605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:23.133488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 832/1687] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:23.238011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9ccc4-0072-4a87-b43f-5e009218f3cf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:23.263797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:23.264427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:23.264707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:23.264982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] PATH_INFO: `/v3/users/d76a04278e0b43738afcddbb13346d8c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:23.278845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76a04278e0b43738afcddbb13346d8c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:23.283567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:23.405792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76a04278e0b43738afcddbb13346d8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:23.407068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76a04278e0b43738afcddbb13346d8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:23.421805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76a04278e0b43738afcddbb13346d8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:23.458158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76a04278e0b43738afcddbb13346d8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:23.477952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-297a96d9-7c0f-4c03-b474-e42e004db54c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76a04278e0b43738afcddbb13346d8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:23.491013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 856/1688] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:23 2022] DELETE /identity/v3/users/d76a04278e0b43738afcddbb13346d8c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:23.495720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5b11-1000-4e44-be17-d606d114bbaf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:23.496011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5b11-1000-4e44-be17-d606d114bbaf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:23.496289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5b11-1000-4e44-be17-d606d114bbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:23.535337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2146e1-9941-4279-a40e-4fc8ccc71e3a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:23.535558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2146e1-9941-4279-a40e-4fc8ccc71e3a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:23.535809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2146e1-9941-4279-a40e-4fc8ccc71e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:23.591458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b6fe5b11-1000-4e44-be17-d606d114bbaf None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:23.595504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b6fe5b11-1000-4e44-be17-d606d114bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:23.630388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5f2146e1-9941-4279-a40e-4fc8ccc71e3a None None] MFA Rules not processed for user `938efacddf794fd18cd323043906b6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:23.631863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5f2146e1-9941-4279-a40e-4fc8ccc71e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:23.734544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 833/1689] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:23.756302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe5b11-1000-4e44-be17-d606d114bbaf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:23.757751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b6fe5b11-1000-4e44-be17-d606d114bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:23.856691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 857/1690] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:23.865657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2146e1-9941-4279-a40e-4fc8ccc71e3a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:23.867762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5f2146e1-9941-4279-a40e-4fc8ccc71e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:23.915516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe5b11-1000-4e44-be17-d606d114bbaf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:23.927332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33426153-cbd1-4f3f-9cd2-07759001348a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:23.928021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33426153-cbd1-4f3f-9cd2-07759001348a service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:23.928304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33426153-cbd1-4f3f-9cd2-07759001348a service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:23.928584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33426153-cbd1-4f3f-9cd2-07759001348a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:23.944645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:23.965366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33426153-cbd1-4f3f-9cd2-07759001348a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:23.971447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33426153-cbd1-4f3f-9cd2-07759001348a service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:23.998920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 834/1691] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:20:24.037980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2146e1-9941-4279-a40e-4fc8ccc71e3a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:24.052922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:24.053790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:24.054250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:24.054551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:24.064664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:24.066499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:24.078651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 858/1692] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:20:24.565620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b761a7-39f9-446f-bb9d-85c4bb48468d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:24.565904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b761a7-39f9-446f-bb9d-85c4bb48468d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:24.567457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b761a7-39f9-446f-bb9d-85c4bb48468d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:24.621493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-17b761a7-39f9-446f-bb9d-85c4bb48468d None None] MFA Rules not processed for user `9b178568bc5e4876bdab388ac3302f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:24.624081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-17b761a7-39f9-446f-bb9d-85c4bb48468d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:24.704546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:24.837920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 835/1693] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:24.919999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:24.921831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:24.963741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d9fff-cde1-4e53-80a6-25cf5d478d4e service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:24.983353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:24.984291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:24.984578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:24.984855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] PATH_INFO: `/v3/projects/084a5b7c3611482a9fa62180ae3f2e9a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:24.991995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=084a5b7c3611482a9fa62180ae3f2e9a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:24.994070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:25.095662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '084a5b7c3611482a9fa62180ae3f2e9a', 'request_id': 'req-45a6a966-7f23-4b45-8f18-5616406dc925'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.102494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '084a5b7c3611482a9fa62180ae3f2e9a', 'request_id': 'req-45a6a966-7f23-4b45-8f18-5616406dc925'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.127115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '084a5b7c3611482a9fa62180ae3f2e9a', 'request_id': 'req-45a6a966-7f23-4b45-8f18-5616406dc925'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.150553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] The token cache is being invalidate because project 084a5b7c3611482a9fa62180ae3f2e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:25.150897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.151628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a6a966-7f23-4b45-8f18-5616406dc925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.155185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 859/1694] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:24 2022] DELETE /identity/v3/projects/084a5b7c3611482a9fa62180ae3f2e9a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:25.160084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b761a7-39f9-446f-bb9d-85c4bb48468d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:25.174028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b761a7-39f9-446f-bb9d-85c4bb48468d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:25.313844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:25.316159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:25.316696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:25.318545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] PATH_INFO: `/v3/users/324bd79b91e647acac91e20f146b358f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:25.320176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324bd79b91e647acac91e20f146b358f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:25.326257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:25.445018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324bd79b91e647acac91e20f146b358f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.459124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324bd79b91e647acac91e20f146b358f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.474764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324bd79b91e647acac91e20f146b358f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.475866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324bd79b91e647acac91e20f146b358f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.502625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b7521b3d-3afb-4038-9515-84347716c7b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324bd79b91e647acac91e20f146b358f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:25.515750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 836/1695] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:25 2022] DELETE /identity/v3/users/324bd79b91e647acac91e20f146b358f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:26.465230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75978c8f-794b-4122-9b3c-ebac6d441397 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:26.465852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75978c8f-794b-4122-9b3c-ebac6d441397 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:26.466307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75978c8f-794b-4122-9b3c-ebac6d441397 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:26.471353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 860/1696] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 23 08:20:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:26.495801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c77d8-1fd1-46c3-9b1d-3ab339b4c36d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:26.496258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c77d8-1fd1-46c3-9b1d-3ab339b4c36d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:26.496365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c77d8-1fd1-46c3-9b1d-3ab339b4c36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:26.541153 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b00c77d8-1fd1-46c3-9b1d-3ab339b4c36d None None] MFA Rules not processed for user `2965b1c8ab52457f8f167acda4ba6844`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:26.542711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b00c77d8-1fd1-46c3-9b1d-3ab339b4c36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:26.638888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:26.654545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45dfe2-c558-4ecd-9100-5ecdad858323 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:26.655020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45dfe2-c558-4ecd-9100-5ecdad858323 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:26.655538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45dfe2-c558-4ecd-9100-5ecdad858323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:26.711523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 837/1697] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:20:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:26.726420 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c77d8-1fd1-46c3-9b1d-3ab339b4c36d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:26.728646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b00c77d8-1fd1-46c3-9b1d-3ab339b4c36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:26.754236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c77d8-1fd1-46c3-9b1d-3ab339b4c36d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:26.762882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9a45dfe2-c558-4ecd-9100-5ecdad858323 None None] MFA Rules not processed for user `eaf926af634c4ec2ada5fc8f4aa95e07`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:26.766562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9a45dfe2-c558-4ecd-9100-5ecdad858323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:26.770146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f45b1-2719-458c-9248-ff35ea1af4bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:26.771078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f45b1-2719-458c-9248-ff35ea1af4bf service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:26.771558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f45b1-2719-458c-9248-ff35ea1af4bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:26.771858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f45b1-2719-458c-9248-ff35ea1af4bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:26.788663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f45b1-2719-458c-9248-ff35ea1af4bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:26.790201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f45b1-2719-458c-9248-ff35ea1af4bf service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:26.811660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 838/1698] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:26.882884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 861/1699] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:26.896448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7309b-a33a-421c-bd5b-bb514c305017 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:26.896760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7309b-a33a-421c-bd5b-bb514c305017 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:26.897574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7309b-a33a-421c-bd5b-bb514c305017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:26.924633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45dfe2-c558-4ecd-9100-5ecdad858323 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:26.928016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9a45dfe2-c558-4ecd-9100-5ecdad858323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:26.939843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-dce7309b-a33a-421c-bd5b-bb514c305017 None None] MFA Rules not processed for user `eaf926af634c4ec2ada5fc8f4aa95e07`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:26.940853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dce7309b-a33a-421c-bd5b-bb514c305017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:26.960873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 839/1700] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:26.994952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:26.995571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7309b-a33a-421c-bd5b-bb514c305017 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:27.000987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dce7309b-a33a-421c-bd5b-bb514c305017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:27.083037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45dfe2-c558-4ecd-9100-5ecdad858323 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:27.090827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:27.096342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:27.096874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:27.097302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] PATH_INFO: `/v3/projects/b31cfb8695144d45bb10b67a69d5c866` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:27.098831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31cfb8695144d45bb10b67a69d5c866)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:27.114490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:27.294961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31cfb8695144d45bb10b67a69d5c866', 'request_id': 'req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:27.324635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7309b-a33a-421c-bd5b-bb514c305017 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:27.331272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31cfb8695144d45bb10b67a69d5c866', 'request_id': 'req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:27.351729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31cfb8695144d45bb10b67a69d5c866', 'request_id': 'req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:27.425075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] The token cache is being invalidate because project b31cfb8695144d45bb10b67a69d5c866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:27.427020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:27.430357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2eb5ae8b-b26c-4523-a2d4-554b5d5c8a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:27.440207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 862/1701] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:26 2022] DELETE /identity/v3/projects/b31cfb8695144d45bb10b67a69d5c866 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:27.447656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea455-87e9-43ea-8ddb-97c607875598 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:27.449333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea455-87e9-43ea-8ddb-97c607875598 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:27.450158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea455-87e9-43ea-8ddb-97c607875598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:27.482615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cf1e7-761d-4611-b486-a7b881a25454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:27.483326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cf1e7-761d-4611-b486-a7b881a25454 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:27.490897 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cf1e7-761d-4611-b486-a7b881a25454 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:27.491462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cf1e7-761d-4611-b486-a7b881a25454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:27.500453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fe0cf1e7-761d-4611-b486-a7b881a25454 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:27.546352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c70ea455-87e9-43ea-8ddb-97c607875598 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:27.556204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c70ea455-87e9-43ea-8ddb-97c607875598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:27.678545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cf1e7-761d-4611-b486-a7b881a25454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:27.683807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cf1e7-761d-4611-b486-a7b881a25454 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:27.724965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 840/1702] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:27.923896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 863/1703] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:27.939258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2009ea-871b-4be6-adad-8a12f986c00d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:27.940211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2009ea-871b-4be6-adad-8a12f986c00d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:27.940942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2009ea-871b-4be6-adad-8a12f986c00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:28.032000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bd2009ea-871b-4be6-adad-8a12f986c00d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:28.034030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bd2009ea-871b-4be6-adad-8a12f986c00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:28.065064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 841/1704] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:28.067743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ea455-87e9-43ea-8ddb-97c607875598 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:28.112608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ea455-87e9-43ea-8ddb-97c607875598 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:28.146178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:28.148984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:28.150606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:28.152749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:28.154053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:28.154723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:28.158154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:28.277814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 864/1705] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:28.284041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2009ea-871b-4be6-adad-8a12f986c00d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:28.320319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2009ea-871b-4be6-adad-8a12f986c00d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:28.344386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a92e2e-632b-4405-aa9b-a05fcfec1e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:28.347940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a92e2e-632b-4405-aa9b-a05fcfec1e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:28.349156 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a92e2e-632b-4405-aa9b-a05fcfec1e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:28.350923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a92e2e-632b-4405-aa9b-a05fcfec1e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:28.354488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a92e2e-632b-4405-aa9b-a05fcfec1e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:28.364161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a92e2e-632b-4405-aa9b-a05fcfec1e75 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:28.382724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:28.526031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 842/1706] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:28 2022] POST /identity/v3/projects => generated 355 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:28.529242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:28.595181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ba8c-1952-448c-9b96-2da32776ef44 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:28.770368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:28.804470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d009-65f5-436f-9a57-127875bb7ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:28.805176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d009-65f5-436f-9a57-127875bb7ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:28.805460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d009-65f5-436f-9a57-127875bb7ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:28.805745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d009-65f5-436f-9a57-127875bb7ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:28.806437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03d009-65f5-436f-9a57-127875bb7ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:28.820405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03d009-65f5-436f-9a57-127875bb7ad9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:28.869930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 865/1707] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:28 2022] POST /identity/v3/users => generated 278 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:28.871023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a92e2e-632b-4405-aa9b-a05fcfec1e75 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:28.924109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a92e2e-632b-4405-aa9b-a05fcfec1e75 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:28.945976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d719b5-2e0a-413c-ac14-c4c991774a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:28.946669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d719b5-2e0a-413c-ac14-c4c991774a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:28.947117 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d719b5-2e0a-413c-ac14-c4c991774a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:28.947560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d719b5-2e0a-413c-ac14-c4c991774a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:28.948460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d719b5-2e0a-413c-ac14-c4c991774a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:28.950401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d719b5-2e0a-413c-ac14-c4c991774a93 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:28.983067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 843/1708] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:28.987497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d009-65f5-436f-9a57-127875bb7ad9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:29.004820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d009-65f5-436f-9a57-127875bb7ad9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:29.021576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ed47-dc59-4c5c-9b43-a6b76c6fc4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:29.022281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ed47-dc59-4c5c-9b43-a6b76c6fc4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:29.022563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ed47-dc59-4c5c-9b43-a6b76c6fc4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:29.022838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ed47-dc59-4c5c-9b43-a6b76c6fc4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:29.028490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559ed47-dc59-4c5c-9b43-a6b76c6fc4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:29.032455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559ed47-dc59-4c5c-9b43-a6b76c6fc4ab admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:29.075492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 866/1709] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:20:29.079415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d719b5-2e0a-413c-ac14-c4c991774a93 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:29.100502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d719b5-2e0a-413c-ac14-c4c991774a93 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:29.126736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbf339-03ad-4e5c-a8e6-fa7758adf80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:29.128585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbf339-03ad-4e5c-a8e6-fa7758adf80d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:29.129101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbf339-03ad-4e5c-a8e6-fa7758adf80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:29.130718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbf339-03ad-4e5c-a8e6-fa7758adf80d admin admin] PATH_INFO: `/v3/projects/d69aca66d3714cd3b351edf578862382/users/1bd42ae5dc0c49e694d5f289dba2e260/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:29.168786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbf339-03ad-4e5c-a8e6-fa7758adf80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69aca66d3714cd3b351edf578862382, user_id=1bd42ae5dc0c49e694d5f289dba2e260, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:29.180185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbf339-03ad-4e5c-a8e6-fa7758adf80d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:29.226495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 844/1710] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:29 2022] PUT /identity/v3/projects/d69aca66d3714cd3b351edf578862382/users/1bd42ae5dc0c49e694d5f289dba2e260/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:29.236062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04335264-df5f-40ad-8986-32dd3d1ee674 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:29.236241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04335264-df5f-40ad-8986-32dd3d1ee674 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:29.236461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04335264-df5f-40ad-8986-32dd3d1ee674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:29.329274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-04335264-df5f-40ad-8986-32dd3d1ee674 None None] MFA Rules not processed for user `1bd42ae5dc0c49e694d5f289dba2e260`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:29.330529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-04335264-df5f-40ad-8986-32dd3d1ee674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:29.557723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 867/1711] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:29 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:29.563388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbf339-03ad-4e5c-a8e6-fa7758adf80d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:29.766926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbf339-03ad-4e5c-a8e6-fa7758adf80d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:29.774017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a68fe2-163f-4466-b1d7-e97790be77d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:29.774223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68fe2-163f-4466-b1d7-e97790be77d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:29.774310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68fe2-163f-4466-b1d7-e97790be77d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:29.774623 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68fe2-163f-4466-b1d7-e97790be77d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:29.775487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a68fe2-163f-4466-b1d7-e97790be77d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:29.779095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a68fe2-163f-4466-b1d7-e97790be77d4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:29.835916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04335264-df5f-40ad-8986-32dd3d1ee674 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:29.839583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 845/1712] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:29 2022] POST /identity/v3/projects => generated 355 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:29.855651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04335264-df5f-40ad-8986-32dd3d1ee674 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:29.991848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4566ac-2366-4d95-8bc4-96e55776667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:30.005235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6ad44-9ee3-42cc-bf64-bcffdce0cc9b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:30.006317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6ad44-9ee3-42cc-bf64-bcffdce0cc9b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:30.006609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6ad44-9ee3-42cc-bf64-bcffdce0cc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:30.011831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4566ac-2366-4d95-8bc4-96e55776667b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:30.012118 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4566ac-2366-4d95-8bc4-96e55776667b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:30.016170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4566ac-2366-4d95-8bc4-96e55776667b admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:30.017171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4566ac-2366-4d95-8bc4-96e55776667b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:30.022239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4566ac-2366-4d95-8bc4-96e55776667b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:30.076041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 868/1713] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:29 2022] POST /identity/v3/users => generated 284 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:20:30.077677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0a80f-aa98-4019-a4f2-626b000e249c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:30.077982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0a80f-aa98-4019-a4f2-626b000e249c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:30.078347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0a80f-aa98-4019-a4f2-626b000e249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:30.131554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cfa6ad44-9ee3-42cc-bf64-bcffdce0cc9b None None] MFA Rules not processed for user `938efacddf794fd18cd323043906b6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:30.137281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cfa6ad44-9ee3-42cc-bf64-bcffdce0cc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:30.182840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-68d0a80f-aa98-4019-a4f2-626b000e249c None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:30.185856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-68d0a80f-aa98-4019-a4f2-626b000e249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:30.335581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 846/1714] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:30.336898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6ad44-9ee3-42cc-bf64-bcffdce0cc9b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:30.340435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:30.369889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6ad44-9ee3-42cc-bf64-bcffdce0cc9b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:30.377047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 869/1715] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:20:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:30.378844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:30.379737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:30.380023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:30.380303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:30.381049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:30.382974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:30.402102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 847/1716] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:30.406852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0a80f-aa98-4019-a4f2-626b000e249c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:30.454011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0a80f-aa98-4019-a4f2-626b000e249c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:30.478814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe6e73-b799-48d8-b5ce-24fb5b003f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:30.482278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe6e73-b799-48d8-b5ce-24fb5b003f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:30.482612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe6e73-b799-48d8-b5ce-24fb5b003f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:30.503494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe6e73-b799-48d8-b5ce-24fb5b003f5d admin admin] PATH_INFO: `/v3/projects/e052c8cd2407487b982485e4e966db24/users/1ee676f6c8de47f08ba6f0bcc179f7e7/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:30.571054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe6e73-b799-48d8-b5ce-24fb5b003f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e052c8cd2407487b982485e4e966db24, user_id=1ee676f6c8de47f08ba6f0bcc179f7e7, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:30.575889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe6e73-b799-48d8-b5ce-24fb5b003f5d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:30.628005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:30.631349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:30.645462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 870/1717] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:30 2022] PUT /identity/v3/projects/e052c8cd2407487b982485e4e966db24/users/1ee676f6c8de47f08ba6f0bcc179f7e7/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:30.649578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe6e73-b799-48d8-b5ce-24fb5b003f5d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:30.715498 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:30.883619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048856e-af5b-439d-8a66-8bdffcca89e2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:30.940184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe6e73-b799-48d8-b5ce-24fb5b003f5d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:30.955057 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa62a82-d8de-46ac-b4af-a1a3953a1b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:30.956079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa62a82-d8de-46ac-b4af-a1a3953a1b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:30.957335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa62a82-d8de-46ac-b4af-a1a3953a1b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:30.958015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa62a82-d8de-46ac-b4af-a1a3953a1b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:30.959025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa62a82-d8de-46ac-b4af-a1a3953a1b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:30.961210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa62a82-d8de-46ac-b4af-a1a3953a1b8c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:30.981869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 871/1718] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:30.983904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa62a82-d8de-46ac-b4af-a1a3953a1b8c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:30.995171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa62a82-d8de-46ac-b4af-a1a3953a1b8c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:31.002914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e991d9e-69b7-45fc-9555-3d459cbb457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:31.004147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e991d9e-69b7-45fc-9555-3d459cbb457a admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:31.004588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e991d9e-69b7-45fc-9555-3d459cbb457a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:31.005030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e991d9e-69b7-45fc-9555-3d459cbb457a admin admin] PATH_INFO: `/v3/projects/e052c8cd2407487b982485e4e966db24/users/1ee676f6c8de47f08ba6f0bcc179f7e7/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:31.008191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e991d9e-69b7-45fc-9555-3d459cbb457a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e052c8cd2407487b982485e4e966db24, user_id=1ee676f6c8de47f08ba6f0bcc179f7e7, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:31.011952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e991d9e-69b7-45fc-9555-3d459cbb457a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:31.028497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 872/1719] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:30 2022] PUT /identity/v3/projects/e052c8cd2407487b982485e4e966db24/users/1ee676f6c8de47f08ba6f0bcc179f7e7/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:31.046040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda29d5-745d-4dc8-8766-5b4c30d6d7f0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:31.047089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda29d5-745d-4dc8-8766-5b4c30d6d7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:31.047769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda29d5-745d-4dc8-8766-5b4c30d6d7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:31.099764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4eda29d5-745d-4dc8-8766-5b4c30d6d7f0 None None] MFA Rules not processed for user `1ee676f6c8de47f08ba6f0bcc179f7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:31.101850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4eda29d5-745d-4dc8-8766-5b4c30d6d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:31.213978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1204666-2ab0-4f07-b9fd-ec56b742f284 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:31.215656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1204666-2ab0-4f07-b9fd-ec56b742f284 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:31.216163 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1204666-2ab0-4f07-b9fd-ec56b742f284 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:31.216632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1204666-2ab0-4f07-b9fd-ec56b742f284 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:31.304042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 873/1720] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:31 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:31.319305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda29d5-745d-4dc8-8766-5b4c30d6d7f0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:31.326134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4eda29d5-745d-4dc8-8766-5b4c30d6d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:31.501443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1204666-2ab0-4f07-b9fd-ec56b742f284 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:31.509408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1204666-2ab0-4f07-b9fd-ec56b742f284 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:31.511611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:31.555297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 848/1721] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:20:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 928 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 08:20:31.575153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda29d5-745d-4dc8-8766-5b4c30d6d7f0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:31.590171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:31.591171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:31.591710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:31.632342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:31.693334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:31.705289 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:31.767983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 874/1722] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:31 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:20:31.934306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1204666-2ab0-4f07-b9fd-ec56b742f284 service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:32.045266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1204666-2ab0-4f07-b9fd-ec56b742f284 service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:32.063725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431287dd-1a6a-4961-abc7-920011315823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:32.064504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431287dd-1a6a-4961-abc7-920011315823 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:32.065671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431287dd-1a6a-4961-abc7-920011315823 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:32.065965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431287dd-1a6a-4961-abc7-920011315823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:32.196906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:32.280542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:32.284684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:32.303259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431287dd-1a6a-4961-abc7-920011315823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:32.307117 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431287dd-1a6a-4961-abc7-920011315823 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:32.355996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 849/1723] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:31 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:32.478685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506b6bc-c85b-4817-87c5-4fd77169ac2b service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:32.497731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:32.499443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:32.501040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:32.501474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] PATH_INFO: `/v3/users/0351e1ed24a74a309dbc26588b39b2d8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:32.504223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0351e1ed24a74a309dbc26588b39b2d8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:32.508653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:32.693341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0351e1ed24a74a309dbc26588b39b2d8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:32.694450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0351e1ed24a74a309dbc26588b39b2d8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:32.716318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0351e1ed24a74a309dbc26588b39b2d8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:32.745536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0351e1ed24a74a309dbc26588b39b2d8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:32.758836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0351e1ed24a74a309dbc26588b39b2d8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:32.771272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 875/1724] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:32 2022] DELETE /identity/v3/users/0351e1ed24a74a309dbc26588b39b2d8 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:32.774493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425a6ec-e39a-4f4b-8b55-a06a3157732d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:32.774792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425a6ec-e39a-4f4b-8b55-a06a3157732d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:32.775074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425a6ec-e39a-4f4b-8b55-a06a3157732d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:32.825301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e425a6ec-e39a-4f4b-8b55-a06a3157732d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:32.828880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e425a6ec-e39a-4f4b-8b55-a06a3157732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:33.151794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 850/1725] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:33.261713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:33.263237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:33.511454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c2933-a818-4bb1-ac6e-23efa9b4e7cb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:33.519476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0158d-0477-4b42-bcaa-3b0a2b7e4453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:33.520248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0158d-0477-4b42-bcaa-3b0a2b7e4453 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:33.520595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0158d-0477-4b42-bcaa-3b0a2b7e4453 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:33.520932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0158d-0477-4b42-bcaa-3b0a2b7e4453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:33.566871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0158d-0477-4b42-bcaa-3b0a2b7e4453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:33.572566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0158d-0477-4b42-bcaa-3b0a2b7e4453 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:33.585177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:33.626301 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 876/1726] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:33.950402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425a6ec-e39a-4f4b-8b55-a06a3157732d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:33.952796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e425a6ec-e39a-4f4b-8b55-a06a3157732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:33.984546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425a6ec-e39a-4f4b-8b55-a06a3157732d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:34.008775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:34.011326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733e76f-bb34-48be-8129-3e5dfe72a98c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:34.011764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733e76f-bb34-48be-8129-3e5dfe72a98c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:34.012557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733e76f-bb34-48be-8129-3e5dfe72a98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:34.034159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:34.039297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:34.044055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] PATH_INFO: `/v3/users/851d6918e1e64933b9bcacc4be364c76` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:34.052773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=851d6918e1e64933b9bcacc4be364c76)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:34.073901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:34.105612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a733e76f-bb34-48be-8129-3e5dfe72a98c None None] MFA Rules not processed for user `2965b1c8ab52457f8f167acda4ba6844`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:34.109603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a733e76f-bb34-48be-8129-3e5dfe72a98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:34.124968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 877/1727] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Jun 23 08:20:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:34.268971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851d6918e1e64933b9bcacc4be364c76'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:34.293037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851d6918e1e64933b9bcacc4be364c76'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:34.321881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851d6918e1e64933b9bcacc4be364c76'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:34.323417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851d6918e1e64933b9bcacc4be364c76'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:34.355406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851d6918e1e64933b9bcacc4be364c76'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:34.377957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 851/1728] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:33 2022] DELETE /identity/v3/users/851d6918e1e64933b9bcacc4be364c76 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:34.380412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330716e-9a3b-4112-a8ef-4d9ed22fd501 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:34.380894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330716e-9a3b-4112-a8ef-4d9ed22fd501 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:34.381329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330716e-9a3b-4112-a8ef-4d9ed22fd501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:34.428080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c330716e-9a3b-4112-a8ef-4d9ed22fd501 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:34.432615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c330716e-9a3b-4112-a8ef-4d9ed22fd501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:34.664408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 878/1729] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:34.853997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:34.855661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:34.862441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330716e-9a3b-4112-a8ef-4d9ed22fd501 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:34.864018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c330716e-9a3b-4112-a8ef-4d9ed22fd501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:35.004430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:35.092188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330716e-9a3b-4112-a8ef-4d9ed22fd501 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:35.102326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89abd54-386f-4bc6-aefc-a32e17528eb6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:35.113224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64739321-6ff7-41cf-873b-67359282152e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:35.116405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:35.117311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64739321-6ff7-41cf-873b-67359282152e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:35.117603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64739321-6ff7-41cf-873b-67359282152e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:35.118570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64739321-6ff7-41cf-873b-67359282152e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:35.118704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:35.118704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:35.119900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:35.131973 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-64739321-6ff7-41cf-873b-67359282152e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:35.162123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:35.168277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:35.208605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64739321-6ff7-41cf-873b-67359282152e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:35.210103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64739321-6ff7-41cf-873b-67359282152e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:35.214881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 852/1730] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:35.245807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 879/1731] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:37.123645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:37.126097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:37.156123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b9437-0227-4a12-a93d-3cf13e63c9b0 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:37.164399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:37.165637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:37.166255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:37.166994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] PATH_INFO: `/v3/users/1ba291bab038457a9d3fa215a2231e54` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:37.168752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ba291bab038457a9d3fa215a2231e54)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:37.171494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:37.306752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ba291bab038457a9d3fa215a2231e54'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:37.325048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ba291bab038457a9d3fa215a2231e54'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:37.354019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ba291bab038457a9d3fa215a2231e54'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:37.356306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ba291bab038457a9d3fa215a2231e54'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:37.406853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ba291bab038457a9d3fa215a2231e54'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:37.430921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 853/1732] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:37 2022] DELETE /identity/v3/users/1ba291bab038457a9d3fa215a2231e54 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:37.432051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f43f2-70f4-42de-9620-6c18d25ae5cb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:37.433001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f43f2-70f4-42de-9620-6c18d25ae5cb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:37.433487 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f43f2-70f4-42de-9620-6c18d25ae5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:37.471305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a60f43f2-70f4-42de-9620-6c18d25ae5cb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:37.478950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a60f43f2-70f4-42de-9620-6c18d25ae5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:37.687684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 880/1733] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:37.969737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:37.971181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:38.122935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb9200-ac50-49f4-b728-0a21cd8b7d63 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:38.137090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:38.137906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:38.138035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:38.138279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:38.159860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:38.161703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:38.190185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 854/1734] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:38.863889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f43f2-70f4-42de-9620-6c18d25ae5cb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:38.865870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a60f43f2-70f4-42de-9620-6c18d25ae5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:38.887935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f43f2-70f4-42de-9620-6c18d25ae5cb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:38.898868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:38.901913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:38.902199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:38.902680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] PATH_INFO: `/v3/projects/007ea29822834890b7bdee37392945ef` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:38.908427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] RBAC: Authorizing `identity:delete_project(project_id=007ea29822834890b7bdee37392945ef)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:38.910402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:38.933951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:38.936122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:38.959465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c573bd-5470-4ef6-a0fd-0fba597b6ab0 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:38.983827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:38.986156 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:38.988693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:38.989226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] PATH_INFO: `/v3/projects/1c8a0a9fab3e42a58d3cb273c536922c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:39.000029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c8a0a9fab3e42a58d3cb273c536922c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:39.002227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:39.039853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:39.104523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '007ea29822834890b7bdee37392945ef', 'request_id': 'req-2aa5f639-869a-46b8-9cfa-0db53aca212a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.129553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '007ea29822834890b7bdee37392945ef', 'request_id': 'req-2aa5f639-869a-46b8-9cfa-0db53aca212a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.147807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '007ea29822834890b7bdee37392945ef', 'request_id': 'req-2aa5f639-869a-46b8-9cfa-0db53aca212a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.177122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] The token cache is being invalidate because project 007ea29822834890b7bdee37392945ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:39.177613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.179314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.183947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 881/1735] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:38 2022] DELETE /identity/v3/projects/007ea29822834890b7bdee37392945ef => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:39.187356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c8a0a9fab3e42a58d3cb273c536922c', 'request_id': 'req-f19ad123-6960-4626-8a46-03322205179b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.189667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:39.190021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:39.192330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:39.197343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c8a0a9fab3e42a58d3cb273c536922c', 'request_id': 'req-f19ad123-6960-4626-8a46-03322205179b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.219356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c8a0a9fab3e42a58d3cb273c536922c', 'request_id': 'req-f19ad123-6960-4626-8a46-03322205179b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.265169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] The token cache is being invalidate because project 1c8a0a9fab3e42a58d3cb273c536922c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:39.266240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.267163 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f19ad123-6960-4626-8a46-03322205179b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.273231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 855/1736] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:38 2022] DELETE /identity/v3/projects/1c8a0a9fab3e42a58d3cb273c536922c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:39.279607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7522444-d5a0-4d87-9a10-65459ffcbddd None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:39.280161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7522444-d5a0-4d87-9a10-65459ffcbddd None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:39.280701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7522444-d5a0-4d87-9a10-65459ffcbddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:39.337182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a7522444-d5a0-4d87-9a10-65459ffcbddd None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:39.338923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a7522444-d5a0-4d87-9a10-65459ffcbddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:39.393332 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f639-869a-46b8-9cfa-0db53aca212a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:39.524014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 856/1737] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:39.532144 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81718d71-bf77-4f30-8bfa-03e8437e46cf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:39.534991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81718d71-bf77-4f30-8bfa-03e8437e46cf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:39.534991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81718d71-bf77-4f30-8bfa-03e8437e46cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:39.590831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:39.591944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:39.592315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:39.592603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] PATH_INFO: `/v3/users/c8746086da8947f68a212a3fdc520dae` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:39.596188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8746086da8947f68a212a3fdc520dae)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:39.600214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:39.652842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-81718d71-bf77-4f30-8bfa-03e8437e46cf None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:39.654381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-81718d71-bf77-4f30-8bfa-03e8437e46cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:39.670100 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 857/1738] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:39.671103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81718d71-bf77-4f30-8bfa-03e8437e46cf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:39.697854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81718d71-bf77-4f30-8bfa-03e8437e46cf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:39.707957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:39.709092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:39.709571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:39.710007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:39.711077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:39.711794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:39.713771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:39.763158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8746086da8947f68a212a3fdc520dae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.764169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8746086da8947f68a212a3fdc520dae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.765140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 858/1739] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:39.766328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:39.778140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8746086da8947f68a212a3fdc520dae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.804049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:39.805688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bae07-686b-48e6-884a-fbc0101eadd0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:39.815999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8746086da8947f68a212a3fdc520dae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.823738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22801ea-501f-4cfb-b969-6be6546640e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:39.824730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22801ea-501f-4cfb-b969-6be6546640e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:39.825214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22801ea-501f-4cfb-b969-6be6546640e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:39.825890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22801ea-501f-4cfb-b969-6be6546640e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:39.830724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22801ea-501f-4cfb-b969-6be6546640e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:39.834402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22801ea-501f-4cfb-b969-6be6546640e0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:39.841450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8746086da8947f68a212a3fdc520dae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:39.858434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 882/1740] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:39 2022] DELETE /identity/v3/users/c8746086da8947f68a212a3fdc520dae => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:39.922804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 859/1741] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:39 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:39.923552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:39.925818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:40.132936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab2e36-7466-4008-96dd-e0cf05b88aa1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:40.242210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3255fa-507d-45ed-9472-48eec8c61c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:40.243056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3255fa-507d-45ed-9472-48eec8c61c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:40.243364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3255fa-507d-45ed-9472-48eec8c61c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:40.243707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3255fa-507d-45ed-9472-48eec8c61c06 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:40.244483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3255fa-507d-45ed-9472-48eec8c61c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:40.247152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3255fa-507d-45ed-9472-48eec8c61c06 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:40.314117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 883/1742] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:39 2022] POST /identity/v3/users => generated 286 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:40.317408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22801ea-501f-4cfb-b969-6be6546640e0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:40.329275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22801ea-501f-4cfb-b969-6be6546640e0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:40.358714 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38669c5-d865-4ee6-877a-d6799309c0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:40.359874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38669c5-d865-4ee6-877a-d6799309c0de admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:40.360397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38669c5-d865-4ee6-877a-d6799309c0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:40.361382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38669c5-d865-4ee6-877a-d6799309c0de admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:40.365157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38669c5-d865-4ee6-877a-d6799309c0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:40.367130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38669c5-d865-4ee6-877a-d6799309c0de admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:40.406481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 860/1743] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:20:40.411235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3255fa-507d-45ed-9472-48eec8c61c06 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:40.425112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3255fa-507d-45ed-9472-48eec8c61c06 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:40.461236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866417f6-b06c-47e2-b2e0-453f9abe6a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:40.462005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866417f6-b06c-47e2-b2e0-453f9abe6a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:40.465396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866417f6-b06c-47e2-b2e0-453f9abe6a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:40.465746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866417f6-b06c-47e2-b2e0-453f9abe6a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:40.467195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866417f6-b06c-47e2-b2e0-453f9abe6a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:40.471232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866417f6-b06c-47e2-b2e0-453f9abe6a20 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:40.523164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 884/1744] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:40.527758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38669c5-d865-4ee6-877a-d6799309c0de admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:40.545536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38669c5-d865-4ee6-877a-d6799309c0de admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:40.553466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:40.555174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:40.555671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:40.556545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] PATH_INFO: `/v3/projects/56e14ca337a749d7a90d794bcc45b01e/users/c77ad5458095401b88b7bf101eadacc7/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:40.592673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e14ca337a749d7a90d794bcc45b01e, user_id=c77ad5458095401b88b7bf101eadacc7, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:40.599404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:40.625886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 861/1745] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:40 2022] PUT /identity/v3/projects/56e14ca337a749d7a90d794bcc45b01e/users/c77ad5458095401b88b7bf101eadacc7/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:40.641508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d124-4b34-4428-a2ed-732a44aa5297 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:40.644912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d124-4b34-4428-a2ed-732a44aa5297 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:40.645437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d124-4b34-4428-a2ed-732a44aa5297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:40.710779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc41d124-4b34-4428-a2ed-732a44aa5297 None None] MFA Rules not processed for user `c77ad5458095401b88b7bf101eadacc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:40.712165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc41d124-4b34-4428-a2ed-732a44aa5297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:40.802941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:40.804684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:40.904804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:40.957842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0498fe-ccf8-4f3e-afe8-40a80173a01f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:40.963935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 885/1746] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:40.970467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:40.972072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:40.972561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:40.973375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] PATH_INFO: `/v3/projects/718f03eb90184025a738418782f8f7b3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:40.974324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41d124-4b34-4428-a2ed-732a44aa5297 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:40.975590 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=718f03eb90184025a738418782f8f7b3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:40.981994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:41.005042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:41.057919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41d124-4b34-4428-a2ed-732a44aa5297 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:41.079358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:41.080280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:41.080664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:41.081176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:41.082102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:41.087299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:41.173263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718f03eb90184025a738418782f8f7b3', 'request_id': 'req-a46ad4cd-1804-42a3-b673-4153526938cc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:41.195643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718f03eb90184025a738418782f8f7b3', 'request_id': 'req-a46ad4cd-1804-42a3-b673-4153526938cc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:41.221847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718f03eb90184025a738418782f8f7b3', 'request_id': 'req-a46ad4cd-1804-42a3-b673-4153526938cc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:41.268165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] The token cache is being invalidate because project 718f03eb90184025a738418782f8f7b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:41.268395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:41.276361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 886/1747] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:40 2022] POST /identity/v3/projects => generated 371 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:41.278872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:41.282813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:41.291004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:41.296393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 862/1748] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:40 2022] DELETE /identity/v3/projects/718f03eb90184025a738418782f8f7b3 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:41.300583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:41.316215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:41.610556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ad4cd-1804-42a3-b673-4153526938cc admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:41.652124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bd8b2-4035-4c97-a897-05ef6e81ca82 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:41.654330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:41.657328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:41.663916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:41.677049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] PATH_INFO: `/v3/users/57b74d81419e4c78a0a687a58d0e96b0` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:41.677935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a19ff-8ab7-49d3-a8c5-fc7f9ec187b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:41.678682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a19ff-8ab7-49d3-a8c5-fc7f9ec187b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:41.678967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a19ff-8ab7-49d3-a8c5-fc7f9ec187b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:41.679265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a19ff-8ab7-49d3-a8c5-fc7f9ec187b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:41.683607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b74d81419e4c78a0a687a58d0e96b0)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:41.688271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a19ff-8ab7-49d3-a8c5-fc7f9ec187b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:41.694630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a19ff-8ab7-49d3-a8c5-fc7f9ec187b1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:41.718396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:41.831244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 887/1749] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:41 2022] POST /identity/v3/users => generated 292 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:41.841098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a19ff-8ab7-49d3-a8c5-fc7f9ec187b1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:41.912495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a19ff-8ab7-49d3-a8c5-fc7f9ec187b1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:41.937671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b74d81419e4c78a0a687a58d0e96b0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:41.953738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7de5-55a4-4d11-95a7-0e2e8856c371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:41.954328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b74d81419e4c78a0a687a58d0e96b0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:41.954539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7de5-55a4-4d11-95a7-0e2e8856c371 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:41.954727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7de5-55a4-4d11-95a7-0e2e8856c371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:41.954984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7de5-55a4-4d11-95a7-0e2e8856c371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:41.961835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c7de5-55a4-4d11-95a7-0e2e8856c371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:41.969211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c7de5-55a4-4d11-95a7-0e2e8856c371 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:41.980953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b74d81419e4c78a0a687a58d0e96b0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:41.981820 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b74d81419e4c78a0a687a58d0e96b0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:42.030286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 888/1750] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:42.036227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7de5-55a4-4d11-95a7-0e2e8856c371 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:42.049785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0e168fea-2a0c-42fb-a823-88a2622fe81c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b74d81419e4c78a0a687a58d0e96b0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:42.062787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7de5-55a4-4d11-95a7-0e2e8856c371 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:42.082281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 863/1751] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:41 2022] DELETE /identity/v3/users/57b74d81419e4c78a0a687a58d0e96b0 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:42.083979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e73295-0c53-4cc1-9dc3-7dafea371847 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.086323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e73295-0c53-4cc1-9dc3-7dafea371847 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.086918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e73295-0c53-4cc1-9dc3-7dafea371847 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.091871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 864/1752] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:20:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:42.149713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f38a-adcc-4c19-85a0-f54c95aa41f3 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.150003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f38a-adcc-4c19-85a0-f54c95aa41f3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.150290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f38a-adcc-4c19-85a0-f54c95aa41f3 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.152169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 865/1753] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:20:42.233406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4a35-9207-414c-a519-502edf68c6e4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.233712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4a35-9207-414c-a519-502edf68c6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.234051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4a35-9207-414c-a519-502edf68c6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.269420 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-32ba4a35-9207-414c-a519-502edf68c6e4 None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:42.273962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-32ba4a35-9207-414c-a519-502edf68c6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:42.305656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:42.307845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:42.308420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.308718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.309251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] PATH_INFO: `/v3/projects/cf39db6c918842a69cf76c0b52a59028/users/e1cd69c0d51f449eaa4168d30bc77084/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.348046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf39db6c918842a69cf76c0b52a59028, user_id=e1cd69c0d51f449eaa4168d30bc77084, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:42.350888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:42.381589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 889/1754] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:42 2022] PUT /identity/v3/projects/cf39db6c918842a69cf76c0b52a59028/users/e1cd69c0d51f449eaa4168d30bc77084/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:42.385805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:42.388888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:42.493575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 866/1755] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:20:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:42.506884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:42.537715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79260f5-109a-4ee3-aa40-27a31abd7b48 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:42.546037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:42.546743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.549928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.550236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.553780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:42.565541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:42.600547 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 890/1756] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:42.603002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba4a35-9207-414c-a519-502edf68c6e4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:42.629723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:42.632598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:42.632871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba4a35-9207-414c-a519-502edf68c6e4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:42.650406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6c113-356b-454b-b9d0-4be043c6c91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:42.651508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c113-356b-454b-b9d0-4be043c6c91f admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.652036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c113-356b-454b-b9d0-4be043c6c91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.652582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c113-356b-454b-b9d0-4be043c6c91f admin admin] PATH_INFO: `/v3/projects/cf39db6c918842a69cf76c0b52a59028/users/e1cd69c0d51f449eaa4168d30bc77084/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.658437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6c113-356b-454b-b9d0-4be043c6c91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf39db6c918842a69cf76c0b52a59028, user_id=e1cd69c0d51f449eaa4168d30bc77084, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:42.661821 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9549ea-eede-4f13-abac-d7a63a6d942d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:42.667326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6c113-356b-454b-b9d0-4be043c6c91f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:42.675979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:42.678102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.678443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.678607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] PATH_INFO: `/v3/projects/2e796e1706e940d59eb7aa54bd48364a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.691736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e796e1706e940d59eb7aa54bd48364a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:42.693879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:42.713716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 867/1757] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:42 2022] PUT /identity/v3/projects/cf39db6c918842a69cf76c0b52a59028/users/e1cd69c0d51f449eaa4168d30bc77084/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:42.727686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906edd-87bd-4424-b208-5c7fde4e2953 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.728218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906edd-87bd-4424-b208-5c7fde4e2953 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.728653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906edd-87bd-4424-b208-5c7fde4e2953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.832625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e6906edd-87bd-4424-b208-5c7fde4e2953 None None] MFA Rules not processed for user `e1cd69c0d51f449eaa4168d30bc77084`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:42.833867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e6906edd-87bd-4424-b208-5c7fde4e2953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:42.859247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e796e1706e940d59eb7aa54bd48364a', 'request_id': 'req-b82bf47e-c038-43be-a34b-481bbe562a8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:42.880912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e796e1706e940d59eb7aa54bd48364a', 'request_id': 'req-b82bf47e-c038-43be-a34b-481bbe562a8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:42.901443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e796e1706e940d59eb7aa54bd48364a', 'request_id': 'req-b82bf47e-c038-43be-a34b-481bbe562a8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:42.933267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] The token cache is being invalidate because project 2e796e1706e940d59eb7aa54bd48364a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:42.934659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:42.935823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82bf47e-c038-43be-a34b-481bbe562a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:42.938324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 891/1758] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:42 2022] DELETE /identity/v3/projects/2e796e1706e940d59eb7aa54bd48364a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:42.940069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a595895-19dd-424f-a7f7-b40c6563beef None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:42.940526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a595895-19dd-424f-a7f7-b40c6563beef None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:42.940985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a595895-19dd-424f-a7f7-b40c6563beef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:42.976428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9a595895-19dd-424f-a7f7-b40c6563beef None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:42.985924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9a595895-19dd-424f-a7f7-b40c6563beef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:43.004873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 868/1759] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:43.005753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6906edd-87bd-4424-b208-5c7fde4e2953 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:43.021482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e6906edd-87bd-4424-b208-5c7fde4e2953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:43.094951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 892/1760] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:20:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:43.097395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ca9a5-4da2-460a-b98c-81d160b93572 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:43.097781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ca9a5-4da2-460a-b98c-81d160b93572 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:43.099242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ca9a5-4da2-460a-b98c-81d160b93572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:43.128046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-639ca9a5-4da2-460a-b98c-81d160b93572 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:43.130175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-639ca9a5-4da2-460a-b98c-81d160b93572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:43.189725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6906edd-87bd-4424-b208-5c7fde4e2953 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:43.195502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:43.196178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:43.196461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:43.196737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:43.198609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:43.203856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:43.217858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:43.220993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:43.238471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 869/1761] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:43.239518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:43.240864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:43.248429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:43.259804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 893/1762] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:43.269088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a327-7498-439e-971b-1b4c6acf47de None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:43.269476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a327-7498-439e-971b-1b4c6acf47de None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:43.269853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a327-7498-439e-971b-1b4c6acf47de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:43.308335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f232a327-7498-439e-971b-1b4c6acf47de None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:43.319501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f232a327-7498-439e-971b-1b4c6acf47de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:43.338398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 894/1763] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:43.340196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232a327-7498-439e-971b-1b4c6acf47de None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:43.359380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232a327-7498-439e-971b-1b4c6acf47de None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:43.369704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60bad1-b51c-45cd-8c2f-cf4873950e82 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:43.379581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470fa58-20f5-4cb4-b6d1-01eff9e6a327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:43.381356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470fa58-20f5-4cb4-b6d1-01eff9e6a327 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:43.381743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470fa58-20f5-4cb4-b6d1-01eff9e6a327 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:43.383360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:43.383650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470fa58-20f5-4cb4-b6d1-01eff9e6a327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:43.384314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:43.386078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:43.387109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:43.391147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:43.399086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:43.401500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:43.406638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470fa58-20f5-4cb4-b6d1-01eff9e6a327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:43.410669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470fa58-20f5-4cb4-b6d1-01eff9e6a327 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:43.427691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 895/1764] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:43.431061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:43.438475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 870/1765] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:43.460475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2d65f-efae-4cb3-99ad-3ab369b665cd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:43.469718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:43.473975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:43.474986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:43.476049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:43.478099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:43.484269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:43.596913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 896/1766] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:43 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:43.597905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470fa58-20f5-4cb4-b6d1-01eff9e6a327 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:43.632662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470fa58-20f5-4cb4-b6d1-01eff9e6a327 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:43.636242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:43.638840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:43.808832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:43.810734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:43.812102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:43.812855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:43.813093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:43.818400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:43.831249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffeae0c-402b-4aca-b012-059da2902f5e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:43.860762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce874f3-f673-47d5-ad26-3911650fef26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:43.864180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce874f3-f673-47d5-ad26-3911650fef26 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:43.864180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce874f3-f673-47d5-ad26-3911650fef26 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:43.864180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce874f3-f673-47d5-ad26-3911650fef26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:43.867624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 871/1767] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:43 2022] POST /identity/v3/users => generated 284 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:43.868558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:43.874522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:43.892575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce874f3-f673-47d5-ad26-3911650fef26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:43.896385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce874f3-f673-47d5-ad26-3911650fef26 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:43.924594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 897/1768] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:43.937319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce874f3-f673-47d5-ad26-3911650fef26 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:43.982588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce874f3-f673-47d5-ad26-3911650fef26 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:44.006571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb315c-5d4c-4bd7-a07f-42b6e3c083fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:44.007765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb315c-5d4c-4bd7-a07f-42b6e3c083fe admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:44.008057 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb315c-5d4c-4bd7-a07f-42b6e3c083fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:44.008337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb315c-5d4c-4bd7-a07f-42b6e3c083fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:44.009117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb315c-5d4c-4bd7-a07f-42b6e3c083fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:44.010789 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb315c-5d4c-4bd7-a07f-42b6e3c083fe admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:44.075258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 898/1769] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:44.076976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb315c-5d4c-4bd7-a07f-42b6e3c083fe admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:44.089478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e99e0-1823-4066-9a0f-c271558a7ef4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:44.102657 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472232bf-9aac-4ebc-9c56-4bf50d1c71f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:44.105401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472232bf-9aac-4ebc-9c56-4bf50d1c71f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:44.107248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472232bf-9aac-4ebc-9c56-4bf50d1c71f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:44.107833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472232bf-9aac-4ebc-9c56-4bf50d1c71f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:44.110357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb315c-5d4c-4bd7-a07f-42b6e3c083fe admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:44.116269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0958f-f613-4a23-88e9-5174c0ecb8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:44.121110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0958f-f613-4a23-88e9-5174c0ecb8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:44.121656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0958f-f613-4a23-88e9-5174c0ecb8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:44.122088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0958f-f613-4a23-88e9-5174c0ecb8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:44.122789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-472232bf-9aac-4ebc-9c56-4bf50d1c71f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:44.123748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0958f-f613-4a23-88e9-5174c0ecb8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:44.127383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0958f-f613-4a23-88e9-5174c0ecb8c6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:44.147344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 899/1770] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:44.153515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0958f-f613-4a23-88e9-5174c0ecb8c6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:44.169308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0958f-f613-4a23-88e9-5174c0ecb8c6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:44.183011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7110ac-e7ae-4c44-ad6e-ae34597c0e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:44.184565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7110ac-e7ae-4c44-ad6e-ae34597c0e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:44.185103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7110ac-e7ae-4c44-ad6e-ae34597c0e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:44.185626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7110ac-e7ae-4c44-ad6e-ae34597c0e26 admin admin] PATH_INFO: `/v3/projects/849921110a0a42d68d41e031b8222e39/users/721479a13a174d168337e4038f882c86/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:44.217956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:44.230451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7110ac-e7ae-4c44-ad6e-ae34597c0e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849921110a0a42d68d41e031b8222e39, user_id=721479a13a174d168337e4038f882c86, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:44.239260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472232bf-9aac-4ebc-9c56-4bf50d1c71f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:44.239767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7110ac-e7ae-4c44-ad6e-ae34597c0e26 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:44.242628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472232bf-9aac-4ebc-9c56-4bf50d1c71f3 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:44.264003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 872/1771] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:44.304607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 900/1772] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:44 2022] PUT /identity/v3/projects/849921110a0a42d68d41e031b8222e39/users/721479a13a174d168337e4038f882c86/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:44.306803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ecfe-4c38-43fc-9d9c-cf2c616dbdd8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:44.307505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ecfe-4c38-43fc-9d9c-cf2c616dbdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:44.307984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ecfe-4c38-43fc-9d9c-cf2c616dbdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:44.363870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8909ecfe-4c38-43fc-9d9c-cf2c616dbdd8 None None] MFA Rules not processed for user `721479a13a174d168337e4038f882c86`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:44.367290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8909ecfe-4c38-43fc-9d9c-cf2c616dbdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:44.559948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 873/1773] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:44.564816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7110ac-e7ae-4c44-ad6e-ae34597c0e26 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:44.697046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:44.742655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7110ac-e7ae-4c44-ad6e-ae34597c0e26 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:44.753853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1d4ee-5d31-485c-876e-fdc3e9e2eeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:44.754993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1d4ee-5d31-485c-876e-fdc3e9e2eeec admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:44.755467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1d4ee-5d31-485c-876e-fdc3e9e2eeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:44.755896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1d4ee-5d31-485c-876e-fdc3e9e2eeec admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:44.756752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1d4ee-5d31-485c-876e-fdc3e9e2eeec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:44.769709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1d4ee-5d31-485c-876e-fdc3e9e2eeec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:44.827141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 901/1774] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:44 2022] POST /identity/v3/projects => generated 367 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:44.838632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909ecfe-4c38-43fc-9d9c-cf2c616dbdd8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:44.860299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909ecfe-4c38-43fc-9d9c-cf2c616dbdd8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:45.011915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722f771-a86f-4719-8b5f-3d0d9addca0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:45.017536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f771-a86f-4719-8b5f-3d0d9addca0a admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:45.017915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f771-a86f-4719-8b5f-3d0d9addca0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:45.018297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f771-a86f-4719-8b5f-3d0d9addca0a admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:45.019999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722f771-a86f-4719-8b5f-3d0d9addca0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:45.022631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722f771-a86f-4719-8b5f-3d0d9addca0a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:45.080900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 874/1775] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:44 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:45.084596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1d4ee-5d31-485c-876e-fdc3e9e2eeec admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:45.102600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1d4ee-5d31-485c-876e-fdc3e9e2eeec admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:45.107644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71f247-1c38-418a-a8a5-3a74af8b8aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:45.108798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71f247-1c38-418a-a8a5-3a74af8b8aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:45.109177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71f247-1c38-418a-a8a5-3a74af8b8aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:45.113205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71f247-1c38-418a-a8a5-3a74af8b8aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:45.113205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71f247-1c38-418a-a8a5-3a74af8b8aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:45.113205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71f247-1c38-418a-a8a5-3a74af8b8aaf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:45.137832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 902/1776] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:20:45.144290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722f771-a86f-4719-8b5f-3d0d9addca0a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:45.158191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722f771-a86f-4719-8b5f-3d0d9addca0a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:45.163788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0cbd0-8f13-4277-8698-f938120a5e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:45.166197 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cbd0-8f13-4277-8698-f938120a5e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:45.166345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cbd0-8f13-4277-8698-f938120a5e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:45.167291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cbd0-8f13-4277-8698-f938120a5e2e admin admin] PATH_INFO: `/v3/projects/8b20c030e62a4fc295ae629749f07530/users/1cdd69d18ecd4d6bb68d09737e74a151/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:45.191537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0cbd0-8f13-4277-8698-f938120a5e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b20c030e62a4fc295ae629749f07530, user_id=1cdd69d18ecd4d6bb68d09737e74a151, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:45.197210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0cbd0-8f13-4277-8698-f938120a5e2e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:45.219993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 875/1777] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:45 2022] PUT /identity/v3/projects/8b20c030e62a4fc295ae629749f07530/users/1cdd69d18ecd4d6bb68d09737e74a151/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:45.225700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71f247-1c38-418a-a8a5-3a74af8b8aaf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:45.236624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71f247-1c38-418a-a8a5-3a74af8b8aaf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:45.300893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39243610-0665-4343-aaa6-0116bf105008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:45.301529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243610-0665-4343-aaa6-0116bf105008 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:45.301811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243610-0665-4343-aaa6-0116bf105008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:45.302088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243610-0665-4343-aaa6-0116bf105008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:45.302874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39243610-0665-4343-aaa6-0116bf105008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:45.304606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39243610-0665-4343-aaa6-0116bf105008 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:45.317304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 903/1778] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:20:45.322529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0cbd0-8f13-4277-8698-f938120a5e2e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:45.333246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0cbd0-8f13-4277-8698-f938120a5e2e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:45.337922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccd69e-3164-4ca6-a7d6-0cf75e5b2786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:45.338891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccd69e-3164-4ca6-a7d6-0cf75e5b2786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:45.339261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccd69e-3164-4ca6-a7d6-0cf75e5b2786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:45.339600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccd69e-3164-4ca6-a7d6-0cf75e5b2786 admin admin] PATH_INFO: `/v3/projects/8b20c030e62a4fc295ae629749f07530/users/1cdd69d18ecd4d6bb68d09737e74a151/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:45.341675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccd69e-3164-4ca6-a7d6-0cf75e5b2786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b20c030e62a4fc295ae629749f07530, user_id=1cdd69d18ecd4d6bb68d09737e74a151, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:45.344548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccd69e-3164-4ca6-a7d6-0cf75e5b2786 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:45.356907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 876/1779] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:45 2022] PUT /identity/v3/projects/8b20c030e62a4fc295ae629749f07530/users/1cdd69d18ecd4d6bb68d09737e74a151/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:45.368358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b653e-ef2a-4b3b-bd50-3895a359ad8c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:45.368776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b653e-ef2a-4b3b-bd50-3895a359ad8c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:45.369063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b653e-ef2a-4b3b-bd50-3895a359ad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:45.398060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c68b653e-ef2a-4b3b-bd50-3895a359ad8c None None] MFA Rules not processed for user `1cdd69d18ecd4d6bb68d09737e74a151`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:45.399438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c68b653e-ef2a-4b3b-bd50-3895a359ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:45.488688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 904/1780] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:45.501279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccd69e-3164-4ca6-a7d6-0cf75e5b2786 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:45.543881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:45.580454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccd69e-3164-4ca6-a7d6-0cf75e5b2786 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:45.584523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833791a7-e879-4253-9e10-20eb41c4e661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:45.585351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833791a7-e879-4253-9e10-20eb41c4e661 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:45.585743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833791a7-e879-4253-9e10-20eb41c4e661 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:45.586101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833791a7-e879-4253-9e10-20eb41c4e661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:45.617825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833791a7-e879-4253-9e10-20eb41c4e661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:45.621282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833791a7-e879-4253-9e10-20eb41c4e661 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:45.633016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 877/1781] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:45.848653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b653e-ef2a-4b3b-bd50-3895a359ad8c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:45.850468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c68b653e-ef2a-4b3b-bd50-3895a359ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:45.986732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:46.033104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b653e-ef2a-4b3b-bd50-3895a359ad8c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:46.043319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:46.045878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:46.048629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:46.049219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] PATH_INFO: `/v3/projects/3f7b3690abc74b5797251b8ae2968e16` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:46.052426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f7b3690abc74b5797251b8ae2968e16)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:46.058934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:46.228326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f7b3690abc74b5797251b8ae2968e16', 'request_id': 'req-fe9b000d-86d2-4210-b030-95f49cb8846a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:46.241828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f7b3690abc74b5797251b8ae2968e16', 'request_id': 'req-fe9b000d-86d2-4210-b030-95f49cb8846a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:46.255697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f7b3690abc74b5797251b8ae2968e16', 'request_id': 'req-fe9b000d-86d2-4210-b030-95f49cb8846a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:46.279675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] The token cache is being invalidate because project 3f7b3690abc74b5797251b8ae2968e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:46.284112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:46.285414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe9b000d-86d2-4210-b030-95f49cb8846a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:46.287648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 905/1782] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:45 2022] DELETE /identity/v3/projects/3f7b3690abc74b5797251b8ae2968e16 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:46.294102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a992685-ab5a-4894-80a5-8b88031086a8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:46.297270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a992685-ab5a-4894-80a5-8b88031086a8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:46.297270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a992685-ab5a-4894-80a5-8b88031086a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:46.321211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6a992685-ab5a-4894-80a5-8b88031086a8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:46.323113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6a992685-ab5a-4894-80a5-8b88031086a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:46.516807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 878/1783] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:46.524758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565946c9-38fd-4500-ab28-389a097c008e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:46.525239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565946c9-38fd-4500-ab28-389a097c008e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:46.525684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565946c9-38fd-4500-ab28-389a097c008e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:46.562123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-565946c9-38fd-4500-ab28-389a097c008e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:46.564591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-565946c9-38fd-4500-ab28-389a097c008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:46.584378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 906/1784] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:46.588480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a992685-ab5a-4894-80a5-8b88031086a8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:46.601603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a992685-ab5a-4894-80a5-8b88031086a8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:46.624199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:46.625122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:46.625406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:46.625727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:46.627199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:46.627854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:46.633200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:46.660654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 879/1785] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:46.666479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565946c9-38fd-4500-ab28-389a097c008e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:46.684668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565946c9-38fd-4500-ab28-389a097c008e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:46.699444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c07251-1210-4345-97ba-9bdc839cb632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:46.700172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c07251-1210-4345-97ba-9bdc839cb632 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:46.700401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c07251-1210-4345-97ba-9bdc839cb632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:46.700679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c07251-1210-4345-97ba-9bdc839cb632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:46.702101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c07251-1210-4345-97ba-9bdc839cb632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:46.703520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c07251-1210-4345-97ba-9bdc839cb632 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:46.770033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 907/1786] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:46 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:46.773996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:46.798414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a368b-9167-4cd5-854c-282d6765aa3a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:46.897778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034fd8a-3cbc-4399-bcb5-8cd8e9257b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:46.898360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034fd8a-3cbc-4399-bcb5-8cd8e9257b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:46.898654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034fd8a-3cbc-4399-bcb5-8cd8e9257b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:46.899634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034fd8a-3cbc-4399-bcb5-8cd8e9257b95 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:46.900466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034fd8a-3cbc-4399-bcb5-8cd8e9257b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:46.902568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034fd8a-3cbc-4399-bcb5-8cd8e9257b95 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:46.944779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 880/1787] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:46 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:46.949771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c07251-1210-4345-97ba-9bdc839cb632 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:46.966307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c07251-1210-4345-97ba-9bdc839cb632 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:46.972084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4fcfc-b942-4783-94f2-1e3686469fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:46.972918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4fcfc-b942-4783-94f2-1e3686469fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:46.973579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4fcfc-b942-4783-94f2-1e3686469fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:46.973579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4fcfc-b942-4783-94f2-1e3686469fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:46.974107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4fcfc-b942-4783-94f2-1e3686469fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:46.975774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4fcfc-b942-4783-94f2-1e3686469fc0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:47.004505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 908/1788] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:47.006554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034fd8a-3cbc-4399-bcb5-8cd8e9257b95 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:47.019908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034fd8a-3cbc-4399-bcb5-8cd8e9257b95 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:47.035970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6f6c9-2aef-4a2b-a9df-c03a02f43361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:47.036674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6f6c9-2aef-4a2b-a9df-c03a02f43361 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:47.036959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6f6c9-2aef-4a2b-a9df-c03a02f43361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:47.037239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6f6c9-2aef-4a2b-a9df-c03a02f43361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:47.038074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6f6c9-2aef-4a2b-a9df-c03a02f43361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:47.039630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6f6c9-2aef-4a2b-a9df-c03a02f43361 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:47.064963 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 881/1789] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:47.066823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4fcfc-b942-4783-94f2-1e3686469fc0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:47.085866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4fcfc-b942-4783-94f2-1e3686469fc0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:47.092919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af621fcd-e5ab-4b9a-a7eb-296ec3f9d90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:47.094189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af621fcd-e5ab-4b9a-a7eb-296ec3f9d90f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:47.094669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af621fcd-e5ab-4b9a-a7eb-296ec3f9d90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:47.095582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af621fcd-e5ab-4b9a-a7eb-296ec3f9d90f admin admin] PATH_INFO: `/v3/projects/29cd4d61477349f594017ea4f237fe60/users/432bd183b26c41fdb6be6a7c86d8de6e/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:47.116390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af621fcd-e5ab-4b9a-a7eb-296ec3f9d90f admin admin] RBAC: Authorizing `identity:create_grant(project_id=29cd4d61477349f594017ea4f237fe60, user_id=432bd183b26c41fdb6be6a7c86d8de6e, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:47.120224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af621fcd-e5ab-4b9a-a7eb-296ec3f9d90f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:47.142222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 909/1790] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:47 2022] PUT /identity/v3/projects/29cd4d61477349f594017ea4f237fe60/users/432bd183b26c41fdb6be6a7c86d8de6e/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:47.153036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797072-ba11-4fd6-9879-c2a529c8af1b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:47.153334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797072-ba11-4fd6-9879-c2a529c8af1b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:47.153615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797072-ba11-4fd6-9879-c2a529c8af1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:47.195196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-09797072-ba11-4fd6-9879-c2a529c8af1b None None] MFA Rules not processed for user `432bd183b26c41fdb6be6a7c86d8de6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:47.196492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-09797072-ba11-4fd6-9879-c2a529c8af1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:47.243969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:47.325243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 882/1791] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:47.330430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af621fcd-e5ab-4b9a-a7eb-296ec3f9d90f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:47.415116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af621fcd-e5ab-4b9a-a7eb-296ec3f9d90f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:47.421713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2b6ac-30db-4b50-b29c-d8ae19b00d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:47.422731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2b6ac-30db-4b50-b29c-d8ae19b00d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:47.423170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2b6ac-30db-4b50-b29c-d8ae19b00d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:47.423615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2b6ac-30db-4b50-b29c-d8ae19b00d3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:47.424657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2b6ac-30db-4b50-b29c-d8ae19b00d3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:47.428707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2b6ac-30db-4b50-b29c-d8ae19b00d3e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:47.448015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:47.474378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 910/1792] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:47 2022] POST /identity/v3/projects => generated 397 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:47.478608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09797072-ba11-4fd6-9879-c2a529c8af1b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:47.497039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09797072-ba11-4fd6-9879-c2a529c8af1b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:47.586634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e9433-1998-4784-beea-85e3fc5438d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:47.587812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9433-1998-4784-beea-85e3fc5438d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:47.588204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9433-1998-4784-beea-85e3fc5438d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:47.588559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9433-1998-4784-beea-85e3fc5438d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:47.589376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e9433-1998-4784-beea-85e3fc5438d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:47.591851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e9433-1998-4784-beea-85e3fc5438d0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:47.621451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 883/1793] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:47 2022] POST /identity/v3/users => generated 305 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:47.625299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2b6ac-30db-4b50-b29c-d8ae19b00d3e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:47.638727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2b6ac-30db-4b50-b29c-d8ae19b00d3e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:47.655691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b779fd-8fa7-4bf4-b77e-f3153790e34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:47.656400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b779fd-8fa7-4bf4-b77e-f3153790e34b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:47.656686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b779fd-8fa7-4bf4-b77e-f3153790e34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:47.656988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b779fd-8fa7-4bf4-b77e-f3153790e34b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:47.659483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b779fd-8fa7-4bf4-b77e-f3153790e34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:47.661202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b779fd-8fa7-4bf4-b77e-f3153790e34b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:47.702833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 911/1794] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:47.706243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e9433-1998-4784-beea-85e3fc5438d0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:47.719356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e9433-1998-4784-beea-85e3fc5438d0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:47.728434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab535b2e-a8a4-4fb2-a56b-de347965a79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:47.730201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535b2e-a8a4-4fb2-a56b-de347965a79b admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:47.731005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535b2e-a8a4-4fb2-a56b-de347965a79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:47.731524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535b2e-a8a4-4fb2-a56b-de347965a79b admin admin] PATH_INFO: `/v3/projects/a8fc25600d774fdcaf99afa5e1813a6e/users/ecf382f819b8491aa81dcaffdfb6cdf4/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:47.761977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab535b2e-a8a4-4fb2-a56b-de347965a79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8fc25600d774fdcaf99afa5e1813a6e, user_id=ecf382f819b8491aa81dcaffdfb6cdf4, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:47.765343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab535b2e-a8a4-4fb2-a56b-de347965a79b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:47.812877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 884/1795] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:47 2022] PUT /identity/v3/projects/a8fc25600d774fdcaf99afa5e1813a6e/users/ecf382f819b8491aa81dcaffdfb6cdf4/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:47.817946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b779fd-8fa7-4bf4-b77e-f3153790e34b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:47.838241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b779fd-8fa7-4bf4-b77e-f3153790e34b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:47.963499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baab98-2b8e-4ad7-a4f0-9f9c3fa018a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:47.964347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baab98-2b8e-4ad7-a4f0-9f9c3fa018a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:47.964635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baab98-2b8e-4ad7-a4f0-9f9c3fa018a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:47.964914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baab98-2b8e-4ad7-a4f0-9f9c3fa018a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:47.965790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baab98-2b8e-4ad7-a4f0-9f9c3fa018a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:47.967906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baab98-2b8e-4ad7-a4f0-9f9c3fa018a2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:47.981433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 912/1796] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:47.986293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab535b2e-a8a4-4fb2-a56b-de347965a79b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:47.998535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab535b2e-a8a4-4fb2-a56b-de347965a79b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:48.004767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:48.004767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:48.004767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:48.005130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] PATH_INFO: `/v3/projects/a8fc25600d774fdcaf99afa5e1813a6e/users/ecf382f819b8491aa81dcaffdfb6cdf4/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:48.007059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8fc25600d774fdcaf99afa5e1813a6e, user_id=ecf382f819b8491aa81dcaffdfb6cdf4, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:48.009785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:48.021276 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 885/1797] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:47 2022] PUT /identity/v3/projects/a8fc25600d774fdcaf99afa5e1813a6e/users/ecf382f819b8491aa81dcaffdfb6cdf4/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:48.033606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae747f83-a1f5-4666-93f7-dcb803a6e0ff None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:48.033788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae747f83-a1f5-4666-93f7-dcb803a6e0ff None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:48.034043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae747f83-a1f5-4666-93f7-dcb803a6e0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:48.062687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ae747f83-a1f5-4666-93f7-dcb803a6e0ff None None] MFA Rules not processed for user `ecf382f819b8491aa81dcaffdfb6cdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:48.064590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ae747f83-a1f5-4666-93f7-dcb803a6e0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:48.153534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 913/1798] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:48.165025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:48.166907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:48.231574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d8242-e3b4-42ce-ba32-3152908c6786 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:48.235156 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:48.235834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:48.236151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:48.236418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:48.245559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:48.246981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:48.257097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 886/1799] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:48.335599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae747f83-a1f5-4666-93f7-dcb803a6e0ff None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:48.345165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae747f83-a1f5-4666-93f7-dcb803a6e0ff None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:48.354602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f12702-969c-400e-9cb2-09a197180e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:48.355436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12702-969c-400e-9cb2-09a197180e6c service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:48.355791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12702-969c-400e-9cb2-09a197180e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:48.356134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12702-969c-400e-9cb2-09a197180e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:48.370551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:48.372024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:48.400368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:48.421695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f12702-969c-400e-9cb2-09a197180e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:48.423124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f12702-969c-400e-9cb2-09a197180e6c service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:48.432153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 914/1800] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:48.438649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6d7a9-6d2a-4b71-a2b7-f383ddf5520f service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:48.443326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:48.444217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:48.444500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:48.444773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] PATH_INFO: `/v3/users/721479a13a174d168337e4038f882c86` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:48.445981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=721479a13a174d168337e4038f882c86)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:48.447957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:48.496027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721479a13a174d168337e4038f882c86'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:48.512898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721479a13a174d168337e4038f882c86'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:48.521655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721479a13a174d168337e4038f882c86'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:48.522358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721479a13a174d168337e4038f882c86'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:48.540945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721479a13a174d168337e4038f882c86'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:48.547868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 887/1801] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:48 2022] DELETE /identity/v3/users/721479a13a174d168337e4038f882c86 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:48.552678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea68ffa-8e1b-4b63-b1a9-ec6f91b5e8bb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:48.552968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea68ffa-8e1b-4b63-b1a9-ec6f91b5e8bb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:48.553248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea68ffa-8e1b-4b63-b1a9-ec6f91b5e8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:48.561078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:48.573112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aea68ffa-8e1b-4b63-b1a9-ec6f91b5e8bb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:48.574458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aea68ffa-8e1b-4b63-b1a9-ec6f91b5e8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:48.644710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 915/1802] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:48.651111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:48.652493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:48.728422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd83cf-6296-4102-a027-f1839ccc84ce admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:48.733709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e939a8-aeb9-4cb1-94f3-394385da3665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:48.734449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e939a8-aeb9-4cb1-94f3-394385da3665 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:48.734799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e939a8-aeb9-4cb1-94f3-394385da3665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:48.735149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e939a8-aeb9-4cb1-94f3-394385da3665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:48.783164 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e939a8-aeb9-4cb1-94f3-394385da3665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:48.786507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e939a8-aeb9-4cb1-94f3-394385da3665 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:48.809818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 888/1803] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:49.148842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea68ffa-8e1b-4b63-b1a9-ec6f91b5e8bb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:49.150907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aea68ffa-8e1b-4b63-b1a9-ec6f91b5e8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:49.163814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea68ffa-8e1b-4b63-b1a9-ec6f91b5e8bb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:49.168486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:49.169462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:49.169679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:49.169981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] PATH_INFO: `/v3/projects/849921110a0a42d68d41e031b8222e39` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:49.171294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849921110a0a42d68d41e031b8222e39)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:49.173203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:49.223914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849921110a0a42d68d41e031b8222e39', 'request_id': 'req-cf803179-facf-4eec-b84b-16916af6b120'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.229551 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849921110a0a42d68d41e031b8222e39', 'request_id': 'req-cf803179-facf-4eec-b84b-16916af6b120'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.238075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849921110a0a42d68d41e031b8222e39', 'request_id': 'req-cf803179-facf-4eec-b84b-16916af6b120'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.249670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] The token cache is being invalidate because project 849921110a0a42d68d41e031b8222e39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:49.249954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.250593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.252184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 916/1804] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:49 2022] DELETE /identity/v3/projects/849921110a0a42d68d41e031b8222e39 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:49.256142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e939a8-aeb9-4cb1-94f3-394385da3665 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:49.265811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e939a8-aeb9-4cb1-94f3-394385da3665 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:49.312133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:49.328740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:49.329733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:49.330015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:49.330289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] PATH_INFO: `/v3/users/1cdd69d18ecd4d6bb68d09737e74a151` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:49.331545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cdd69d18ecd4d6bb68d09737e74a151)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:49.333473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:49.387292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cdd69d18ecd4d6bb68d09737e74a151'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.396082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cdd69d18ecd4d6bb68d09737e74a151'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.403707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cdd69d18ecd4d6bb68d09737e74a151'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.404852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cdd69d18ecd4d6bb68d09737e74a151'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.416655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92449cc4-592e-4c97-8f37-06713dc7ee00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cdd69d18ecd4d6bb68d09737e74a151'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.422862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 889/1805] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:49 2022] DELETE /identity/v3/users/1cdd69d18ecd4d6bb68d09737e74a151 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:49.697637 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:49.700005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:49.713896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803179-facf-4eec-b84b-16916af6b120 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:49.776684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:49.776684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:49.776684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:49.776684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] PATH_INFO: `/v3/projects/8b20c030e62a4fc295ae629749f07530` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:49.777840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b20c030e62a4fc295ae629749f07530)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:49.780674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:49.822342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:49.836253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b20c030e62a4fc295ae629749f07530', 'request_id': 'req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.843185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b20c030e62a4fc295ae629749f07530', 'request_id': 'req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.853190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b20c030e62a4fc295ae629749f07530', 'request_id': 'req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.866383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] The token cache is being invalidate because project 8b20c030e62a4fc295ae629749f07530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:49.866673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.867357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e156e40f-2a53-483b-bdb3-1ee911e4e4fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:49.868922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 917/1806] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:49 2022] DELETE /identity/v3/projects/8b20c030e62a4fc295ae629749f07530 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:49.874264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bab37-796d-4cc2-a677-5e29a6b37e48 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:49.874547 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bab37-796d-4cc2-a677-5e29a6b37e48 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:49.874818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bab37-796d-4cc2-a677-5e29a6b37e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:49.893620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-db9bab37-796d-4cc2-a677-5e29a6b37e48 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:49.894910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-db9bab37-796d-4cc2-a677-5e29a6b37e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:49.968530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 890/1807] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:49.976420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092cb08-d133-4a1a-952e-b25f5b21840e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:49.977542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092cb08-d133-4a1a-952e-b25f5b21840e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:49.977924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092cb08-d133-4a1a-952e-b25f5b21840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:49.998440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9092cb08-d133-4a1a-952e-b25f5b21840e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:50.000053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9092cb08-d133-4a1a-952e-b25f5b21840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:50.008148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 918/1808] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:50.012331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bab37-796d-4cc2-a677-5e29a6b37e48 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.022668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bab37-796d-4cc2-a677-5e29a6b37e48 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.027740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.028408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.028716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.029091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.029999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:50.030307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.032070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.044245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 891/1809] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:50.049276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092cb08-d133-4a1a-952e-b25f5b21840e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.058899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092cb08-d133-4a1a-952e-b25f5b21840e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.064117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928482a7-19c5-42d7-9ac9-dda65b92a884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.064857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928482a7-19c5-42d7-9ac9-dda65b92a884 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.065213 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928482a7-19c5-42d7-9ac9-dda65b92a884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.065554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928482a7-19c5-42d7-9ac9-dda65b92a884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.066333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928482a7-19c5-42d7-9ac9-dda65b92a884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.077907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928482a7-19c5-42d7-9ac9-dda65b92a884 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.103836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 919/1810] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:50 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:20:50.108115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.117919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43f66a-6ef3-4b68-b60e-60ab94a81509 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.169876 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:50.181794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896bcc-2547-4692-84f3-0a65d5f409f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.182526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896bcc-2547-4692-84f3-0a65d5f409f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.183259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896bcc-2547-4692-84f3-0a65d5f409f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.183259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896bcc-2547-4692-84f3-0a65d5f409f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.183810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896bcc-2547-4692-84f3-0a65d5f409f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.185794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896bcc-2547-4692-84f3-0a65d5f409f2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.207653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 892/1811] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:50 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:50.211615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928482a7-19c5-42d7-9ac9-dda65b92a884 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.222521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928482a7-19c5-42d7-9ac9-dda65b92a884 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.227287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402146ca-4ba4-453b-b6fb-0be190960fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.227993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402146ca-4ba4-453b-b6fb-0be190960fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.228277 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402146ca-4ba4-453b-b6fb-0be190960fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.228549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402146ca-4ba4-453b-b6fb-0be190960fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.229278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402146ca-4ba4-453b-b6fb-0be190960fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.230881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402146ca-4ba4-453b-b6fb-0be190960fc0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.242662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 920/1812] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:50.247297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896bcc-2547-4692-84f3-0a65d5f409f2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.259239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896bcc-2547-4692-84f3-0a65d5f409f2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.268415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f2e76-63ad-4398-978b-fc644bbd6ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.269201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2e76-63ad-4398-978b-fc644bbd6ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.269553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2e76-63ad-4398-978b-fc644bbd6ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.269898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2e76-63ad-4398-978b-fc644bbd6ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.270702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f2e76-63ad-4398-978b-fc644bbd6ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.272514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f2e76-63ad-4398-978b-fc644bbd6ba8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.284384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 893/1813] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:50.288371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402146ca-4ba4-453b-b6fb-0be190960fc0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.298878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402146ca-4ba4-453b-b6fb-0be190960fc0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.303592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea4c23-425d-4bda-9cd3-31dba3cc40fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.304499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4c23-425d-4bda-9cd3-31dba3cc40fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.304782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4c23-425d-4bda-9cd3-31dba3cc40fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.305062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4c23-425d-4bda-9cd3-31dba3cc40fb admin admin] PATH_INFO: `/v3/projects/c0729253ca104268932e3d686d9fe79b/users/5b02b37a0a3741f79b72cf8593a93c51/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.320473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea4c23-425d-4bda-9cd3-31dba3cc40fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0729253ca104268932e3d686d9fe79b, user_id=5b02b37a0a3741f79b72cf8593a93c51, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.323166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea4c23-425d-4bda-9cd3-31dba3cc40fb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.340755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 921/1814] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:50 2022] PUT /identity/v3/projects/c0729253ca104268932e3d686d9fe79b/users/5b02b37a0a3741f79b72cf8593a93c51/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:50.352166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a74e8-48f1-4820-a770-488c971b0dd7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.352551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a74e8-48f1-4820-a770-488c971b0dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.352895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a74e8-48f1-4820-a770-488c971b0dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.380667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2a7a74e8-48f1-4820-a770-488c971b0dd7 None None] MFA Rules not processed for user `5b02b37a0a3741f79b72cf8593a93c51`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:50.381869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2a7a74e8-48f1-4820-a770-488c971b0dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:50.448845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 894/1815] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:50.453520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea4c23-425d-4bda-9cd3-31dba3cc40fb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.534885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea4c23-425d-4bda-9cd3-31dba3cc40fb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.540132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ac4b4-52ff-494b-8239-c0e455eb3c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.540958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ac4b4-52ff-494b-8239-c0e455eb3c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.541254 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ac4b4-52ff-494b-8239-c0e455eb3c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.541565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ac4b4-52ff-494b-8239-c0e455eb3c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.542609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ac4b4-52ff-494b-8239-c0e455eb3c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.545461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ac4b4-52ff-494b-8239-c0e455eb3c30 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.579772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 922/1816] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:50 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:50.583995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a74e8-48f1-4820-a770-488c971b0dd7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.596892 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a74e8-48f1-4820-a770-488c971b0dd7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.655938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ca53e-195b-4dc2-a19b-50dff0581f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.656621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ca53e-195b-4dc2-a19b-50dff0581f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.656901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ca53e-195b-4dc2-a19b-50dff0581f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.657177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ca53e-195b-4dc2-a19b-50dff0581f74 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.657870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ca53e-195b-4dc2-a19b-50dff0581f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.659835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ca53e-195b-4dc2-a19b-50dff0581f74 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.679226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 895/1817] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:50 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:20:50.683232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ac4b4-52ff-494b-8239-c0e455eb3c30 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.694041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ac4b4-52ff-494b-8239-c0e455eb3c30 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.698767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96383c-0337-43d3-8413-1b67fd2d6c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.699532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96383c-0337-43d3-8413-1b67fd2d6c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.699875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96383c-0337-43d3-8413-1b67fd2d6c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.700210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96383c-0337-43d3-8413-1b67fd2d6c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.701196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96383c-0337-43d3-8413-1b67fd2d6c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.702879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96383c-0337-43d3-8413-1b67fd2d6c7d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.713864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 923/1818] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:50.719045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ca53e-195b-4dc2-a19b-50dff0581f74 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.729652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ca53e-195b-4dc2-a19b-50dff0581f74 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.734755 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b533f9-6e0d-4d9f-be76-d976b4ac2f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.735697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b533f9-6e0d-4d9f-be76-d976b4ac2f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.735975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b533f9-6e0d-4d9f-be76-d976b4ac2f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.736276 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b533f9-6e0d-4d9f-be76-d976b4ac2f4d admin admin] PATH_INFO: `/v3/projects/00fe1f950e654ce19f7a266607fa0844/users/18767dbf04ab4624bdd9213dfc58af25/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.752345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b533f9-6e0d-4d9f-be76-d976b4ac2f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fe1f950e654ce19f7a266607fa0844, user_id=18767dbf04ab4624bdd9213dfc58af25, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.755282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b533f9-6e0d-4d9f-be76-d976b4ac2f4d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.771045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 896/1819] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:50 2022] PUT /identity/v3/projects/00fe1f950e654ce19f7a266607fa0844/users/18767dbf04ab4624bdd9213dfc58af25/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:50.775232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96383c-0337-43d3-8413-1b67fd2d6c7d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.786292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96383c-0337-43d3-8413-1b67fd2d6c7d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.820461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:50.853499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f300967-172c-44a3-ade1-e3d62e220f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.854177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f300967-172c-44a3-ade1-e3d62e220f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.854464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f300967-172c-44a3-ade1-e3d62e220f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.854765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f300967-172c-44a3-ade1-e3d62e220f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.855587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f300967-172c-44a3-ade1-e3d62e220f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.857337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f300967-172c-44a3-ade1-e3d62e220f0a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.868443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 924/1820] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:50.873337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b533f9-6e0d-4d9f-be76-d976b4ac2f4d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:50.884678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b533f9-6e0d-4d9f-be76-d976b4ac2f4d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:50.890001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:50.890907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.891191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.891498 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] PATH_INFO: `/v3/projects/00fe1f950e654ce19f7a266607fa0844/users/18767dbf04ab4624bdd9213dfc58af25/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.893553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fe1f950e654ce19f7a266607fa0844, user_id=18767dbf04ab4624bdd9213dfc58af25, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:50.896417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:50.907314 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 897/1821] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:50 2022] PUT /identity/v3/projects/00fe1f950e654ce19f7a266607fa0844/users/18767dbf04ab4624bdd9213dfc58af25/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:50.917779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c60a3-e5fb-4542-a03e-2ad193239a06 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:50.918100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c60a3-e5fb-4542-a03e-2ad193239a06 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:50.918381 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c60a3-e5fb-4542-a03e-2ad193239a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:50.945429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f85c60a3-e5fb-4542-a03e-2ad193239a06 None None] MFA Rules not processed for user `18767dbf04ab4624bdd9213dfc58af25`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:50.946615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f85c60a3-e5fb-4542-a03e-2ad193239a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:51.017638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 925/1822] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:51.029205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:51.031074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:51.045892 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:51.095983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462b2a2-555f-49ae-8d55-3941e9491919 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:51.104119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59f9e8-6631-4964-aa68-cc82b58a1b0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:51.104789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59f9e8-6631-4964-aa68-cc82b58a1b0b service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:51.105070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59f9e8-6631-4964-aa68-cc82b58a1b0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:51.105343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59f9e8-6631-4964-aa68-cc82b58a1b0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:51.115855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59f9e8-6631-4964-aa68-cc82b58a1b0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:51.117298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59f9e8-6631-4964-aa68-cc82b58a1b0b service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:51.127391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 898/1823] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:51.154368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c60a3-e5fb-4542-a03e-2ad193239a06 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:51.219641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c60a3-e5fb-4542-a03e-2ad193239a06 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:51.227709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:51.228844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:51.229144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:51.229389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] PATH_INFO: `/v3/users/5b02b37a0a3741f79b72cf8593a93c51` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:51.230594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b02b37a0a3741f79b72cf8593a93c51)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:51.232710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:51.282014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b02b37a0a3741f79b72cf8593a93c51'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.282663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b02b37a0a3741f79b72cf8593a93c51'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.287097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:51.290289 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b02b37a0a3741f79b72cf8593a93c51'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.302987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b02b37a0a3741f79b72cf8593a93c51'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.311173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b02b37a0a3741f79b72cf8593a93c51'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.317196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 926/1824] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:51 2022] DELETE /identity/v3/users/5b02b37a0a3741f79b72cf8593a93c51 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:51.321877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841b496-bf20-4f58-9051-a008a96ba7eb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:51.322167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841b496-bf20-4f58-9051-a008a96ba7eb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:51.322447 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841b496-bf20-4f58-9051-a008a96ba7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:51.340696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d841b496-bf20-4f58-9051-a008a96ba7eb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:51.342007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d841b496-bf20-4f58-9051-a008a96ba7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:51.407099 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 899/1825] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:51.416420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:51.418073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:51.481500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea84ea-556b-493e-b3f9-0fb75ccbbb59 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:51.492760 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d8a08-18f4-4035-94a4-99bff8a2ffa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:51.493556 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8a08-18f4-4035-94a4-99bff8a2ffa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:51.493901 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8a08-18f4-4035-94a4-99bff8a2ffa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:51.494235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8a08-18f4-4035-94a4-99bff8a2ffa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:51.505929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d8a08-18f4-4035-94a4-99bff8a2ffa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:51.507727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d8a08-18f4-4035-94a4-99bff8a2ffa9 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:51.519508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 927/1826] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:51.816174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841b496-bf20-4f58-9051-a008a96ba7eb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:51.817595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d841b496-bf20-4f58-9051-a008a96ba7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:51.830658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841b496-bf20-4f58-9051-a008a96ba7eb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:51.836028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:51.836954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:51.837237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:51.837513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] PATH_INFO: `/v3/projects/c0729253ca104268932e3d686d9fe79b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:51.839389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0729253ca104268932e3d686d9fe79b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:51.841394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:51.894407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0729253ca104268932e3d686d9fe79b', 'request_id': 'req-778a63d7-35f1-4807-ba96-7a75a136a00f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.900340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0729253ca104268932e3d686d9fe79b', 'request_id': 'req-778a63d7-35f1-4807-ba96-7a75a136a00f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.909404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0729253ca104268932e3d686d9fe79b', 'request_id': 'req-778a63d7-35f1-4807-ba96-7a75a136a00f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.922059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] The token cache is being invalidate because project c0729253ca104268932e3d686d9fe79b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:51.922345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.923022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:51.927813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 900/1827] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:51 2022] DELETE /identity/v3/projects/c0729253ca104268932e3d686d9fe79b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:51.934918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d8a08-18f4-4035-94a4-99bff8a2ffa9 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:51.944494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d8a08-18f4-4035-94a4-99bff8a2ffa9 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:52.004233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:52.005501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:52.005883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:52.006174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] PATH_INFO: `/v3/users/18767dbf04ab4624bdd9213dfc58af25` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:52.008119 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18767dbf04ab4624bdd9213dfc58af25)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:52.011049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:52.063277 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18767dbf04ab4624bdd9213dfc58af25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.063931 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18767dbf04ab4624bdd9213dfc58af25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.071133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18767dbf04ab4624bdd9213dfc58af25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.084634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18767dbf04ab4624bdd9213dfc58af25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.086368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:52.095061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2def6ef-5361-4192-ad2e-605168dd4b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18767dbf04ab4624bdd9213dfc58af25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.102692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 928/1828] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:51 2022] DELETE /identity/v3/users/18767dbf04ab4624bdd9213dfc58af25 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:52.405173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:52.407523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:52.432328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a63d7-35f1-4807-ba96-7a75a136a00f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:52.478613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:52.510296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:52.512015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:52.512236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:52.512510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] PATH_INFO: `/v3/projects/00fe1f950e654ce19f7a266607fa0844` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:52.513778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00fe1f950e654ce19f7a266607fa0844)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:52.515762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:52.564505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00fe1f950e654ce19f7a266607fa0844', 'request_id': 'req-943363f5-7d13-42b8-af8a-47ad170c8951'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.576437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00fe1f950e654ce19f7a266607fa0844', 'request_id': 'req-943363f5-7d13-42b8-af8a-47ad170c8951'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.585777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00fe1f950e654ce19f7a266607fa0844', 'request_id': 'req-943363f5-7d13-42b8-af8a-47ad170c8951'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.598193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] The token cache is being invalidate because project 00fe1f950e654ce19f7a266607fa0844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:52.598477 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.599129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-943363f5-7d13-42b8-af8a-47ad170c8951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:52.603370 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 901/1829] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:52 2022] DELETE /identity/v3/projects/00fe1f950e654ce19f7a266607fa0844 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:52.609044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ae1f9-4a57-4af5-9f43-225794155aae None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:52.609745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ae1f9-4a57-4af5-9f43-225794155aae None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:52.610026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ae1f9-4a57-4af5-9f43-225794155aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:52.628519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-255ae1f9-4a57-4af5-9f43-225794155aae None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:52.629708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-255ae1f9-4a57-4af5-9f43-225794155aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:52.700866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 929/1830] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:52.708627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb10e31-2064-4b12-858e-abf352f6816a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:52.708968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb10e31-2064-4b12-858e-abf352f6816a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:52.709253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb10e31-2064-4b12-858e-abf352f6816a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:52.729496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-beb10e31-2064-4b12-858e-abf352f6816a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:52.730720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-beb10e31-2064-4b12-858e-abf352f6816a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:52.738751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 902/1831] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:52.743158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ae1f9-4a57-4af5-9f43-225794155aae None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:52.754894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ae1f9-4a57-4af5-9f43-225794155aae None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:52.760740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:52.761469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:52.761696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:52.762070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:52.763946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:52.764265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:52.765891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:52.783293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 930/1832] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:52.785592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb10e31-2064-4b12-858e-abf352f6816a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:52.797292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb10e31-2064-4b12-858e-abf352f6816a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:52.803875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f6e3-8b61-404d-ab0e-1a86b91ddbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:52.804718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f6e3-8b61-404d-ab0e-1a86b91ddbdc admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:52.805017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f6e3-8b61-404d-ab0e-1a86b91ddbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:52.805271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f6e3-8b61-404d-ab0e-1a86b91ddbdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:52.806058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275f6e3-8b61-404d-ab0e-1a86b91ddbdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:52.808013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275f6e3-8b61-404d-ab0e-1a86b91ddbdc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:52.837244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 903/1833] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:52 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:20:52.839184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:52.849501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed42337-8a49-4dd1-8f99-76ca6c6d575d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:52.909748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e207ce-5a08-4ce9-ba10-582e7c65ada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:52.910428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e207ce-5a08-4ce9-ba10-582e7c65ada5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:52.910709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e207ce-5a08-4ce9-ba10-582e7c65ada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:52.910981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e207ce-5a08-4ce9-ba10-582e7c65ada5 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:52.911717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e207ce-5a08-4ce9-ba10-582e7c65ada5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:52.913803 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e207ce-5a08-4ce9-ba10-582e7c65ada5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:52.946194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 931/1834] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:52 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:52.947341 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f6e3-8b61-404d-ab0e-1a86b91ddbdc admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:52.957812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f6e3-8b61-404d-ab0e-1a86b91ddbdc admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:52.963461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14beac7-145b-44cd-b414-dcf5a9f89d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:52.964298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14beac7-145b-44cd-b414-dcf5a9f89d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:52.964581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14beac7-145b-44cd-b414-dcf5a9f89d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:52.964852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14beac7-145b-44cd-b414-dcf5a9f89d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:52.965620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14beac7-145b-44cd-b414-dcf5a9f89d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:52.967298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14beac7-145b-44cd-b414-dcf5a9f89d56 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:52.978708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 904/1835] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:52.982752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e207ce-5a08-4ce9-ba10-582e7c65ada5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:52.993398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e207ce-5a08-4ce9-ba10-582e7c65ada5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:52.999355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922c921-d981-46ac-8869-aac312df61be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:53.000077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922c921-d981-46ac-8869-aac312df61be admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:53.000360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922c921-d981-46ac-8869-aac312df61be admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:53.000650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922c921-d981-46ac-8869-aac312df61be admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:53.001368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922c921-d981-46ac-8869-aac312df61be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:53.003265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922c921-d981-46ac-8869-aac312df61be admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:53.014513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 932/1836] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:53.018817 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14beac7-145b-44cd-b414-dcf5a9f89d56 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:53.030293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14beac7-145b-44cd-b414-dcf5a9f89d56 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:53.035004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:53.035954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:53.036212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:53.036500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] PATH_INFO: `/v3/projects/3722dba444a14413a0c8e3155a56e437/users/9d2fb7f24bf8495e937a1c353cc3cf83/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:53.051918 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3722dba444a14413a0c8e3155a56e437, user_id=9d2fb7f24bf8495e937a1c353cc3cf83, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:53.054633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:53.074481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 905/1837] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:53 2022] PUT /identity/v3/projects/3722dba444a14413a0c8e3155a56e437/users/9d2fb7f24bf8495e937a1c353cc3cf83/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:53.086938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27b48-7bc4-4dad-bc28-96b40d55b250 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:53.087261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27b48-7bc4-4dad-bc28-96b40d55b250 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:53.087548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27b48-7bc4-4dad-bc28-96b40d55b250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:53.116403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f6d27b48-7bc4-4dad-bc28-96b40d55b250 None None] MFA Rules not processed for user `9d2fb7f24bf8495e937a1c353cc3cf83`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:53.117721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f6d27b48-7bc4-4dad-bc28-96b40d55b250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:53.180903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:53.215415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 933/1838] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:53.226115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:53.228206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:53.289160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc41d8-aa42-4aa8-b6af-f9951fb9eae5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:53.293553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd71d8-3eef-4c57-8ec3-ab3bf65adc30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:53.294339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd71d8-3eef-4c57-8ec3-ab3bf65adc30 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:53.294938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd71d8-3eef-4c57-8ec3-ab3bf65adc30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:53.298743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd71d8-3eef-4c57-8ec3-ab3bf65adc30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:53.310511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd71d8-3eef-4c57-8ec3-ab3bf65adc30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:53.311941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd71d8-3eef-4c57-8ec3-ab3bf65adc30 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:53.321709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 906/1839] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:54.315549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d27b48-7bc4-4dad-bc28-96b40d55b250 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:54.317084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f6d27b48-7bc4-4dad-bc28-96b40d55b250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:54.384295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d27b48-7bc4-4dad-bc28-96b40d55b250 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:54.389540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:54.390417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:54.390697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:54.390970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] PATH_INFO: `/v3/users/1bd42ae5dc0c49e694d5f289dba2e260` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:54.392203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd42ae5dc0c49e694d5f289dba2e260)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:54.394211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:54.444495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd42ae5dc0c49e694d5f289dba2e260'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:54.445189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd42ae5dc0c49e694d5f289dba2e260'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:54.452271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd42ae5dc0c49e694d5f289dba2e260'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:54.463628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd42ae5dc0c49e694d5f289dba2e260'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:54.473170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd42ae5dc0c49e694d5f289dba2e260'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:54.479359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 934/1840] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:54 2022] DELETE /identity/v3/users/1bd42ae5dc0c49e694d5f289dba2e260 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:54.485273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef96d3-97a5-465c-b990-1db87318d7b8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:54.485791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef96d3-97a5-465c-b990-1db87318d7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:54.485929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef96d3-97a5-465c-b990-1db87318d7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:54.507247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a0ef96d3-97a5-465c-b990-1db87318d7b8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:54.508480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a0ef96d3-97a5-465c-b990-1db87318d7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:54.512853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:54.575845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 907/1841] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:54.585713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:54.587401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:54.648222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82651cd-67db-471b-b610-0330d71aa6d8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:54.652456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b757a57-99fe-49fd-b749-10fd0951dda1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:54.653102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757a57-99fe-49fd-b749-10fd0951dda1 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:54.653385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757a57-99fe-49fd-b749-10fd0951dda1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:54.653656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757a57-99fe-49fd-b749-10fd0951dda1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:54.663165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b757a57-99fe-49fd-b749-10fd0951dda1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:54.664668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b757a57-99fe-49fd-b749-10fd0951dda1 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:54.674423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 935/1842] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:55.032748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef96d3-97a5-465c-b990-1db87318d7b8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:55.034433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a0ef96d3-97a5-465c-b990-1db87318d7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:55.048774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef96d3-97a5-465c-b990-1db87318d7b8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:55.054312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:55.055124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:55.055452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:55.055734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] PATH_INFO: `/v3/projects/d69aca66d3714cd3b351edf578862382` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:55.063506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69aca66d3714cd3b351edf578862382)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:55.065521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:55.161105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69aca66d3714cd3b351edf578862382', 'request_id': 'req-1d88dd8a-b5aa-4986-b894-19bdf444de1e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.170260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69aca66d3714cd3b351edf578862382', 'request_id': 'req-1d88dd8a-b5aa-4986-b894-19bdf444de1e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.189610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69aca66d3714cd3b351edf578862382', 'request_id': 'req-1d88dd8a-b5aa-4986-b894-19bdf444de1e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.218863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] The token cache is being invalidate because project d69aca66d3714cd3b351edf578862382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:55.220453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.223353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.226313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 908/1843] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:55 2022] DELETE /identity/v3/projects/d69aca66d3714cd3b351edf578862382 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:55.230130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b757a57-99fe-49fd-b749-10fd0951dda1 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:55.244227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b757a57-99fe-49fd-b749-10fd0951dda1 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:55.253867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:55.350925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:55.351992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:55.352274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:55.352552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] PATH_INFO: `/v3/users/1ee676f6c8de47f08ba6f0bcc179f7e7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:55.353764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee676f6c8de47f08ba6f0bcc179f7e7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:55.355739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:55.406747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:55.409358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:55.429242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88dd8a-b5aa-4986-b894-19bdf444de1e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:55.433077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee676f6c8de47f08ba6f0bcc179f7e7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.435756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee676f6c8de47f08ba6f0bcc179f7e7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.441218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:55.442165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:55.442473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:55.442747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] PATH_INFO: `/v3/users/432bd183b26c41fdb6be6a7c86d8de6e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:55.444676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=432bd183b26c41fdb6be6a7c86d8de6e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:55.446094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee676f6c8de47f08ba6f0bcc179f7e7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.447223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:55.476236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee676f6c8de47f08ba6f0bcc179f7e7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.493367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-78199e7a-83cf-4946-b776-110cc089de8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee676f6c8de47f08ba6f0bcc179f7e7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.504958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 936/1844] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:55 2022] DELETE /identity/v3/users/1ee676f6c8de47f08ba6f0bcc179f7e7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:55.534641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '432bd183b26c41fdb6be6a7c86d8de6e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.544677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '432bd183b26c41fdb6be6a7c86d8de6e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.553681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '432bd183b26c41fdb6be6a7c86d8de6e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.554688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '432bd183b26c41fdb6be6a7c86d8de6e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.573006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '432bd183b26c41fdb6be6a7c86d8de6e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.583431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 909/1845] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:55 2022] DELETE /identity/v3/users/432bd183b26c41fdb6be6a7c86d8de6e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:55.585702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039b932-6fad-4a7c-93ac-f040e5e9ea29 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:55.586544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039b932-6fad-4a7c-93ac-f040e5e9ea29 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:55.586544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039b932-6fad-4a7c-93ac-f040e5e9ea29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:55.605909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e039b932-6fad-4a7c-93ac-f040e5e9ea29 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:55.607132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e039b932-6fad-4a7c-93ac-f040e5e9ea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:55.683744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 937/1846] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:55.690861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:55.692551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:55.699347 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:55.753987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8208e-b595-4ef0-b818-a578c82647c0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:55.758010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de881c-c7ee-4af3-a31a-cee91af54c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:55.758663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de881c-c7ee-4af3-a31a-cee91af54c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:55.758936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de881c-c7ee-4af3-a31a-cee91af54c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:55.759261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de881c-c7ee-4af3-a31a-cee91af54c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:55.768891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de881c-c7ee-4af3-a31a-cee91af54c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:55.770482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de881c-c7ee-4af3-a31a-cee91af54c39 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:55.780529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 910/1847] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:55.805246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039b932-6fad-4a7c-93ac-f040e5e9ea29 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:55.806913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e039b932-6fad-4a7c-93ac-f040e5e9ea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:55.822868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039b932-6fad-4a7c-93ac-f040e5e9ea29 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:55.829633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:55.830786 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:55.831802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:55.831802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] PATH_INFO: `/v3/projects/e052c8cd2407487b982485e4e966db24` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:55.832841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e052c8cd2407487b982485e4e966db24)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:55.835063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:55.863980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:55.889516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e052c8cd2407487b982485e4e966db24', 'request_id': 'req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.895112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e052c8cd2407487b982485e4e966db24', 'request_id': 'req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.904412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e052c8cd2407487b982485e4e966db24', 'request_id': 'req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.915875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] The token cache is being invalidate because project e052c8cd2407487b982485e4e966db24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:55.916162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.916818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ceeae162-9a05-4e01-baa6-43fe09dbbb8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:55.918402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 938/1848] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:55 2022] DELETE /identity/v3/projects/e052c8cd2407487b982485e4e966db24 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:55.923485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ff34-bdb9-4aa3-8bcd-a93c771a5ad6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:55.923853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ff34-bdb9-4aa3-8bcd-a93c771a5ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:55.924203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ff34-bdb9-4aa3-8bcd-a93c771a5ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:55.944187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3838ff34-bdb9-4aa3-8bcd-a93c771a5ad6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:55.945397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3838ff34-bdb9-4aa3-8bcd-a93c771a5ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:56.007016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 911/1849] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:56.015506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172681a-67e1-45d0-be17-4d52a5f312b2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.015897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172681a-67e1-45d0-be17-4d52a5f312b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.016245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172681a-67e1-45d0-be17-4d52a5f312b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.035492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b172681a-67e1-45d0-be17-4d52a5f312b2 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:56.036680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b172681a-67e1-45d0-be17-4d52a5f312b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:56.046857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 939/1850] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:56.053066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838ff34-bdb9-4aa3-8bcd-a93c771a5ad6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.063327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838ff34-bdb9-4aa3-8bcd-a93c771a5ad6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.068588 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.069451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.069790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.070145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.070485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172681a-67e1-45d0-be17-4d52a5f312b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.071026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:56.071411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.072295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b172681a-67e1-45d0-be17-4d52a5f312b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:56.073158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.085927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172681a-67e1-45d0-be17-4d52a5f312b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.087700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 912/1851] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:20:56.091721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.092513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.092668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.092761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.093116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] PATH_INFO: `/v3/projects/29cd4d61477349f594017ea4f237fe60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.094268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29cd4d61477349f594017ea4f237fe60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.096241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.105483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10a95f-8052-48d1-9eb1-d65f666670ea admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.111109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c172e-6361-4def-8a03-792c34d640de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.111836 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c172e-6361-4def-8a03-792c34d640de admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.112114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c172e-6361-4def-8a03-792c34d640de admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.112653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c172e-6361-4def-8a03-792c34d640de admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.113464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c172e-6361-4def-8a03-792c34d640de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.115924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c172e-6361-4def-8a03-792c34d640de admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.148070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29cd4d61477349f594017ea4f237fe60', 'request_id': 'req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.150615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 913/1852] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:56 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:20:56.154401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29cd4d61477349f594017ea4f237fe60', 'request_id': 'req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.154785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c172e-6361-4def-8a03-792c34d640de admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.163471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29cd4d61477349f594017ea4f237fe60', 'request_id': 'req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.178604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] The token cache is being invalidate because project 29cd4d61477349f594017ea4f237fe60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:56.179305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.179618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.184319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 940/1853] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:56 2022] DELETE /identity/v3/projects/29cd4d61477349f594017ea4f237fe60 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:56.185370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.186728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:56.229101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c172e-6361-4def-8a03-792c34d640de admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.234729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.235508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.235723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.235996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.236699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.238626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.257779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8354bb-c1a1-460c-9d70-ad6db6cb9b12 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.261699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 914/1854] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:56 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:20:56.263457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.264379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.264635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.265602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.266234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] PATH_INFO: `/v3/users/ecf382f819b8491aa81dcaffdfb6cdf4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.267796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf382f819b8491aa81dcaffdfb6cdf4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.268148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:56.269726 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.276186 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:56.287832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4107f-cfd8-45f8-925b-182575830d67 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.294970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d17f3-6921-4097-95f4-5f4e1d7e4fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.295850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d17f3-6921-4097-95f4-5f4e1d7e4fea admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.296283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d17f3-6921-4097-95f4-5f4e1d7e4fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.296571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d17f3-6921-4097-95f4-5f4e1d7e4fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.297316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d17f3-6921-4097-95f4-5f4e1d7e4fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.299025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d17f3-6921-4097-95f4-5f4e1d7e4fea admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.310935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 915/1855] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:56.314996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d17f3-6921-4097-95f4-5f4e1d7e4fea admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.315797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:56.325906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d17f3-6921-4097-95f4-5f4e1d7e4fea admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.330796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18dc09-2172-4eb3-90e5-5caaf95b023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.331597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18dc09-2172-4eb3-90e5-5caaf95b023b admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.331883 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18dc09-2172-4eb3-90e5-5caaf95b023b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.332162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18dc09-2172-4eb3-90e5-5caaf95b023b admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.332580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf382f819b8491aa81dcaffdfb6cdf4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.332930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18dc09-2172-4eb3-90e5-5caaf95b023b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.333687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf382f819b8491aa81dcaffdfb6cdf4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.334545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18dc09-2172-4eb3-90e5-5caaf95b023b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.341704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf382f819b8491aa81dcaffdfb6cdf4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.345254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 916/1856] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:56.349502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18dc09-2172-4eb3-90e5-5caaf95b023b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.354963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf382f819b8491aa81dcaffdfb6cdf4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.360821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18dc09-2172-4eb3-90e5-5caaf95b023b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.365835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-52b7bfa1-21ad-48bf-83da-4ceeabd50ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf382f819b8491aa81dcaffdfb6cdf4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.366791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.367696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.367981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.368253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] PATH_INFO: `/v3/projects/67b5cf2365d54485b7d5cd648ec83520/users/d3b55c5c38f24a1c9533e1879e9497c1/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.373611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 941/1857] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:20:56 2022] DELETE /identity/v3/users/ecf382f819b8491aa81dcaffdfb6cdf4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:56.386111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b5cf2365d54485b7d5cd648ec83520, user_id=d3b55c5c38f24a1c9533e1879e9497c1, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.389064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.405348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 917/1858] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:56 2022] PUT /identity/v3/projects/67b5cf2365d54485b7d5cd648ec83520/users/d3b55c5c38f24a1c9533e1879e9497c1/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:56.416475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eec9a9-0752-4720-acc2-9b2ce11b747b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.417016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eec9a9-0752-4720-acc2-9b2ce11b747b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.417364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eec9a9-0752-4720-acc2-9b2ce11b747b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.449039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-29eec9a9-0752-4720-acc2-9b2ce11b747b None None] MFA Rules not processed for user `d3b55c5c38f24a1c9533e1879e9497c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:56.450775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-29eec9a9-0752-4720-acc2-9b2ce11b747b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:56.564060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 942/1859] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:56.565212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.567229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:56.646633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1de965-cafa-4e97-87b7-3e51e22887da admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.651621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97657c-b0f3-46ed-9863-2f2ef00789b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.652507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97657c-b0f3-46ed-9863-2f2ef00789b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.652849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97657c-b0f3-46ed-9863-2f2ef00789b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.653185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97657c-b0f3-46ed-9863-2f2ef00789b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.654238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97657c-b0f3-46ed-9863-2f2ef00789b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.656256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97657c-b0f3-46ed-9863-2f2ef00789b4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.690743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 918/1860] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:56 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:56.695629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eec9a9-0752-4720-acc2-9b2ce11b747b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.711249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eec9a9-0752-4720-acc2-9b2ce11b747b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.787993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.788457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.788693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.788972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.789763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.796760 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.818987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 943/1861] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:56 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:56.823174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97657c-b0f3-46ed-9863-2f2ef00789b4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.832508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.834339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:56.836692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97657c-b0f3-46ed-9863-2f2ef00789b4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.844922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7fa97-84f0-4199-96df-b350d406e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.846391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7fa97-84f0-4199-96df-b350d406e4de admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.846391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7fa97-84f0-4199-96df-b350d406e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.846391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7fa97-84f0-4199-96df-b350d406e4de admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.847087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7fa97-84f0-4199-96df-b350d406e4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.849541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7fa97-84f0-4199-96df-b350d406e4de admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.850481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bce8-f444-4d9e-86ce-b0a0c3e38b99 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.857097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.858437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.858701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.858971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] PATH_INFO: `/v3/projects/a8fc25600d774fdcaf99afa5e1813a6e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.860866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8fc25600d774fdcaf99afa5e1813a6e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.862820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.868231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 919/1862] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:56.872382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7fa97-84f0-4199-96df-b350d406e4de admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.883373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7fa97-84f0-4199-96df-b350d406e4de admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.888883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:56.891662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4874e-a11b-46c7-a8dc-8f51479fe3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:56.892739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4874e-a11b-46c7-a8dc-8f51479fe3df admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.893003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4874e-a11b-46c7-a8dc-8f51479fe3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.893619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4874e-a11b-46c7-a8dc-8f51479fe3df admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.894480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4874e-a11b-46c7-a8dc-8f51479fe3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:56.901944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4874e-a11b-46c7-a8dc-8f51479fe3df admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:56.916395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 920/1863] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:20:56.921434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8fc25600d774fdcaf99afa5e1813a6e', 'request_id': 'req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.921776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4874e-a11b-46c7-a8dc-8f51479fe3df admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:56.927138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8fc25600d774fdcaf99afa5e1813a6e', 'request_id': 'req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.933123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4874e-a11b-46c7-a8dc-8f51479fe3df admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:56.937053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8fc25600d774fdcaf99afa5e1813a6e', 'request_id': 'req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.951030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] The token cache is being invalidate because project a8fc25600d774fdcaf99afa5e1813a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:20:56.951575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.953171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3ba83b2-89e7-4fcb-874c-c8e83cd9d0a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:20:56.954986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 944/1864] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:20:56 2022] DELETE /identity/v3/projects/a8fc25600d774fdcaf99afa5e1813a6e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:56.961792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae189943-8596-49c9-bae4-d5d5a5cb041c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:56.962079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae189943-8596-49c9-bae4-d5d5a5cb041c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:56.962359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae189943-8596-49c9-bae4-d5d5a5cb041c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:56.980064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ae189943-8596-49c9-bae4-d5d5a5cb041c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:56.982255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ae189943-8596-49c9-bae4-d5d5a5cb041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:57.014322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8a0eb-9467-4189-bdd2-415c85cf572c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.015871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a0eb-9467-4189-bdd2-415c85cf572c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.016155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a0eb-9467-4189-bdd2-415c85cf572c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.016432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a0eb-9467-4189-bdd2-415c85cf572c admin admin] PATH_INFO: `/v3/projects/74e802ca024e4f46889b17288c6ed79c/users/92540a87b643498fb5845eda53d6b2f6/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.034467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8a0eb-9467-4189-bdd2-415c85cf572c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e802ca024e4f46889b17288c6ed79c, user_id=92540a87b643498fb5845eda53d6b2f6, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.037270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8a0eb-9467-4189-bdd2-415c85cf572c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.051728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 921/1865] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:56 2022] PUT /identity/v3/projects/74e802ca024e4f46889b17288c6ed79c/users/92540a87b643498fb5845eda53d6b2f6/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:57.061071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 945/1866] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:57.061206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c47a7-97a2-4074-90ef-cb99c8af0000 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.061305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c47a7-97a2-4074-90ef-cb99c8af0000 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.061578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c47a7-97a2-4074-90ef-cb99c8af0000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.063153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcb2d-d574-4c9a-9a32-040364f16ab6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.063887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcb2d-d574-4c9a-9a32-040364f16ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.064185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcb2d-d574-4c9a-9a32-040364f16ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.071158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:57.087115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1a8c47a7-97a2-4074-90ef-cb99c8af0000 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:57.088443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1a8c47a7-97a2-4074-90ef-cb99c8af0000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:57.106110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-832bcb2d-d574-4c9a-9a32-040364f16ab6 None None] MFA Rules not processed for user `92540a87b643498fb5845eda53d6b2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:57.107799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-832bcb2d-d574-4c9a-9a32-040364f16ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:57.160676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 922/1867] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:57.164781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c47a7-97a2-4074-90ef-cb99c8af0000 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.175639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c47a7-97a2-4074-90ef-cb99c8af0000 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.182006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.182676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.182951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.183259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.184005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:20:57.184308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.186523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.191182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 946/1868] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:57.192781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bcb2d-d574-4c9a-9a32-040364f16ab6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.194909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-832bcb2d-d574-4c9a-9a32-040364f16ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:57.205639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 923/1869] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:20:57.206788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.215780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bcb2d-d574-4c9a-9a32-040364f16ab6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.219067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ed4d3-d154-425f-874b-1e825393bae1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.222003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7ef2a-d548-404b-ae50-5235b74921b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.222687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7ef2a-d548-404b-ae50-5235b74921b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.222972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7ef2a-d548-404b-ae50-5235b74921b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.223273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7ef2a-d548-404b-ae50-5235b74921b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.223979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7ef2a-d548-404b-ae50-5235b74921b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.225963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7ef2a-d548-404b-ae50-5235b74921b5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.226224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d069cec-073f-462a-b53b-4bd26653409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.226929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069cec-073f-462a-b53b-4bd26653409d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.227231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069cec-073f-462a-b53b-4bd26653409d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.227521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069cec-073f-462a-b53b-4bd26653409d admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.228229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d069cec-073f-462a-b53b-4bd26653409d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.230247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d069cec-073f-462a-b53b-4bd26653409d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.260585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 947/1870] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:57.265271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7ef2a-d548-404b-ae50-5235b74921b5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.265959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 924/1871] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/projects => generated 409 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:57.272179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d069cec-073f-462a-b53b-4bd26653409d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.307031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:57.358688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d069cec-073f-462a-b53b-4bd26653409d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.365915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9e956-6f06-4af8-9483-3b4411d9715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.367705 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e956-6f06-4af8-9483-3b4411d9715a admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.368024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e956-6f06-4af8-9483-3b4411d9715a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.368342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e956-6f06-4af8-9483-3b4411d9715a admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.370055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9e956-6f06-4af8-9483-3b4411d9715a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.375264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9e956-6f06-4af8-9483-3b4411d9715a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.375875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7ef2a-d548-404b-ae50-5235b74921b5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.384225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b18d98-f85e-4bf0-9756-b35894619e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.385406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b18d98-f85e-4bf0-9756-b35894619e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.386220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b18d98-f85e-4bf0-9756-b35894619e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.386669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b18d98-f85e-4bf0-9756-b35894619e84 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.387555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b18d98-f85e-4bf0-9756-b35894619e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.390063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b18d98-f85e-4bf0-9756-b35894619e84 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.398740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 925/1872] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/users => generated 305 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:57.402930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9e956-6f06-4af8-9483-3b4411d9715a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.416292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 948/1873] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:20:57.418459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9e956-6f06-4af8-9483-3b4411d9715a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.420377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b18d98-f85e-4bf0-9756-b35894619e84 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.428368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b84e0-4983-44b2-93be-937252803dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.429123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b84e0-4983-44b2-93be-937252803dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.429408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b84e0-4983-44b2-93be-937252803dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.430702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b84e0-4983-44b2-93be-937252803dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.431542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b84e0-4983-44b2-93be-937252803dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.433289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b84e0-4983-44b2-93be-937252803dd3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.436736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b18d98-f85e-4bf0-9756-b35894619e84 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.442484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272baea-081a-40b8-8e89-fd80fd921e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.443166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272baea-081a-40b8-8e89-fd80fd921e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.443475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272baea-081a-40b8-8e89-fd80fd921e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.443750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272baea-081a-40b8-8e89-fd80fd921e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.444484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272baea-081a-40b8-8e89-fd80fd921e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.446106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272baea-081a-40b8-8e89-fd80fd921e29 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.452240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 926/1874] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:57.453581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b84e0-4983-44b2-93be-937252803dd3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.464151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 949/1875] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:20:57.465919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272baea-081a-40b8-8e89-fd80fd921e29 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.466715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b84e0-4983-44b2-93be-937252803dd3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.475364 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d91bff-0b3b-445f-92e0-01656bae3255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.475871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91bff-0b3b-445f-92e0-01656bae3255 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.476937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91bff-0b3b-445f-92e0-01656bae3255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.477032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272baea-081a-40b8-8e89-fd80fd921e29 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.477154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91bff-0b3b-445f-92e0-01656bae3255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.477154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d91bff-0b3b-445f-92e0-01656bae3255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.478778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d91bff-0b3b-445f-92e0-01656bae3255 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.486482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2e90c-13e1-43e2-b7ff-66e99b2eab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.488795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2e90c-13e1-43e2-b7ff-66e99b2eab6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.489277 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2e90c-13e1-43e2-b7ff-66e99b2eab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.489841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2e90c-13e1-43e2-b7ff-66e99b2eab6f admin admin] PATH_INFO: `/v3/projects/11fd062d57a34558bb4690516fb9c53a/users/bd307c04c65a469e9f34d4fc995e31c9/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.492229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 927/1876] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:20:57.496991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d91bff-0b3b-445f-92e0-01656bae3255 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.514415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d91bff-0b3b-445f-92e0-01656bae3255 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.520010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6396bc-12fc-4210-b910-b88c948c4617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.520952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6396bc-12fc-4210-b910-b88c948c4617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.521214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6396bc-12fc-4210-b910-b88c948c4617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.521479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6396bc-12fc-4210-b910-b88c948c4617 admin admin] PATH_INFO: `/v3/projects/89ae95bbd5aa44a897eec9810c122e7d/users/be04cce95bd94949b752c6ba92c95d8d/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.528423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2e90c-13e1-43e2-b7ff-66e99b2eab6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fd062d57a34558bb4690516fb9c53a, user_id=bd307c04c65a469e9f34d4fc995e31c9, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.531479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2e90c-13e1-43e2-b7ff-66e99b2eab6f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.546125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6396bc-12fc-4210-b910-b88c948c4617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ae95bbd5aa44a897eec9810c122e7d, user_id=be04cce95bd94949b752c6ba92c95d8d, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.549215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6396bc-12fc-4210-b910-b88c948c4617 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.549666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 950/1877] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:57 2022] PUT /identity/v3/projects/11fd062d57a34558bb4690516fb9c53a/users/bd307c04c65a469e9f34d4fc995e31c9/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:57.554032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2e90c-13e1-43e2-b7ff-66e99b2eab6f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.571140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 928/1878] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:57 2022] PUT /identity/v3/projects/89ae95bbd5aa44a897eec9810c122e7d/users/be04cce95bd94949b752c6ba92c95d8d/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:57.588337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53527c41-59a3-4c84-bfbc-05780079428f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.588654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53527c41-59a3-4c84-bfbc-05780079428f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.588955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53527c41-59a3-4c84-bfbc-05780079428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.617541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:57.634637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-53527c41-59a3-4c84-bfbc-05780079428f None None] MFA Rules not processed for user `be04cce95bd94949b752c6ba92c95d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:57.634637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-53527c41-59a3-4c84-bfbc-05780079428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:57.665209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2e90c-13e1-43e2-b7ff-66e99b2eab6f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.723369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 929/1879] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:57.728293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53527c41-59a3-4c84-bfbc-05780079428f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.749484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22a59-51d0-4950-bcce-53c764c22561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.750819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22a59-51d0-4950-bcce-53c764c22561 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.751175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22a59-51d0-4950-bcce-53c764c22561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.751543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22a59-51d0-4950-bcce-53c764c22561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.752341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a22a59-51d0-4950-bcce-53c764c22561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.754736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a22a59-51d0-4950-bcce-53c764c22561 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.766909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 951/1880] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:20:57.777996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22a59-51d0-4950-bcce-53c764c22561 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.798644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22a59-51d0-4950-bcce-53c764c22561 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.803748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cf81b-efbe-4b2f-92fc-a61b4fd614af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.805603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cf81b-efbe-4b2f-92fc-a61b4fd614af admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.807047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cf81b-efbe-4b2f-92fc-a61b4fd614af admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.807341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cf81b-efbe-4b2f-92fc-a61b4fd614af admin admin] PATH_INFO: `/v3/projects/11fd062d57a34558bb4690516fb9c53a/users/bd307c04c65a469e9f34d4fc995e31c9/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.810010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cf81b-efbe-4b2f-92fc-a61b4fd614af admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fd062d57a34558bb4690516fb9c53a, user_id=bd307c04c65a469e9f34d4fc995e31c9, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.812801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cf81b-efbe-4b2f-92fc-a61b4fd614af admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.813493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53527c41-59a3-4c84-bfbc-05780079428f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.822038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6932-2db5-43ce-acef-3c6d7a719796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.822869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6932-2db5-43ce-acef-3c6d7a719796 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.823162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6932-2db5-43ce-acef-3c6d7a719796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.823469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6932-2db5-43ce-acef-3c6d7a719796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.824308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e6932-2db5-43ce-acef-3c6d7a719796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.826453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e6932-2db5-43ce-acef-3c6d7a719796 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.828241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 952/1881] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:57 2022] PUT /identity/v3/projects/11fd062d57a34558bb4690516fb9c53a/users/bd307c04c65a469e9f34d4fc995e31c9/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:57.839373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eef32-420e-4d8c-b07d-bc48a16eb60d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.839867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eef32-420e-4d8c-b07d-bc48a16eb60d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.840599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eef32-420e-4d8c-b07d-bc48a16eb60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.849882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:57.865312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 930/1882] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/projects => generated 409 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:20:57.869229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6932-2db5-43ce-acef-3c6d7a719796 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.883661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-832eef32-420e-4d8c-b07d-bc48a16eb60d None None] MFA Rules not processed for user `bd307c04c65a469e9f34d4fc995e31c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:57.885300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-832eef32-420e-4d8c-b07d-bc48a16eb60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:57.943265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6932-2db5-43ce-acef-3c6d7a719796 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.949532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d0b5c-d9c9-4208-8b72-8a93fc878578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.949947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0b5c-d9c9-4208-8b72-8a93fc878578 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.950317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0b5c-d9c9-4208-8b72-8a93fc878578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.950738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0b5c-d9c9-4208-8b72-8a93fc878578 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.951468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d0b5c-d9c9-4208-8b72-8a93fc878578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.953743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d0b5c-d9c9-4208-8b72-8a93fc878578 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:57.970871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 953/1883] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:20:57.978692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eef32-420e-4d8c-b07d-bc48a16eb60d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.979131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 931/1884] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:20:57 2022] POST /identity/v3/users => generated 311 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:20:57.980143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d0b5c-d9c9-4208-8b72-8a93fc878578 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:57.980268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-832eef32-420e-4d8c-b07d-bc48a16eb60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:57.990126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d0b5c-d9c9-4208-8b72-8a93fc878578 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:57.994942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3c2f-f723-4cfe-a0ba-aea6cee45b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:57.995659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3c2f-f723-4cfe-a0ba-aea6cee45b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:57.995941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3c2f-f723-4cfe-a0ba-aea6cee45b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:57.996217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3c2f-f723-4cfe-a0ba-aea6cee45b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:57.996950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf3c2f-f723-4cfe-a0ba-aea6cee45b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:57.998916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf3c2f-f723-4cfe-a0ba-aea6cee45b98 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:58.010352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 932/1885] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:58.014566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3c2f-f723-4cfe-a0ba-aea6cee45b98 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:58.020889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:58.025198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3c2f-f723-4cfe-a0ba-aea6cee45b98 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:58.031563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d4a51-018c-4576-8690-d6a167c9825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:58.032466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d4a51-018c-4576-8690-d6a167c9825c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:58.032750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d4a51-018c-4576-8690-d6a167c9825c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:58.033028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d4a51-018c-4576-8690-d6a167c9825c admin admin] PATH_INFO: `/v3/projects/a78444d6606140808d457fc6016729e6/users/77b0f7f6364445058d061e28b16f9f4a/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:58.040149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eef32-420e-4d8c-b07d-bc48a16eb60d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:58.043799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfe656-1904-47cc-99ae-a52c0d3b7a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:58.044533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfe656-1904-47cc-99ae-a52c0d3b7a14 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:58.044804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfe656-1904-47cc-99ae-a52c0d3b7a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:58.045076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfe656-1904-47cc-99ae-a52c0d3b7a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:58.048881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d4a51-018c-4576-8690-d6a167c9825c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78444d6606140808d457fc6016729e6, user_id=77b0f7f6364445058d061e28b16f9f4a, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:58.051778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d4a51-018c-4576-8690-d6a167c9825c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:58.054657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfe656-1904-47cc-99ae-a52c0d3b7a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:58.056155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfe656-1904-47cc-99ae-a52c0d3b7a14 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:58.066335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 933/1886] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:58 2022] PUT /identity/v3/projects/a78444d6606140808d457fc6016729e6/users/77b0f7f6364445058d061e28b16f9f4a/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:20:58.067373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 954/1887] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:20:58.070269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d4a51-018c-4576-8690-d6a167c9825c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:58.107685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfe656-1904-47cc-99ae-a52c0d3b7a14 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:58.117041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfe656-1904-47cc-99ae-a52c0d3b7a14 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:58.143888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d4a51-018c-4576-8690-d6a167c9825c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:58.148945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5b853-17c9-427b-83be-da3baae5ad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:58.149725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b853-17c9-427b-83be-da3baae5ad6d admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:58.150007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b853-17c9-427b-83be-da3baae5ad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:58.150278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b853-17c9-427b-83be-da3baae5ad6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:58.151151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5b853-17c9-427b-83be-da3baae5ad6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:58.152772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5b853-17c9-427b-83be-da3baae5ad6d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:58.163227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 934/1888] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:20:58.167313 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5b853-17c9-427b-83be-da3baae5ad6d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:58.170679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:58.171375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:58.171661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:58.171938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:58.174454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:58.177173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5b853-17c9-427b-83be-da3baae5ad6d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:58.181952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fc79e-53a7-462e-9ec2-bb0af2899ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:58.182827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fc79e-53a7-462e-9ec2-bb0af2899ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:58.183110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fc79e-53a7-462e-9ec2-bb0af2899ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:58.183414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fc79e-53a7-462e-9ec2-bb0af2899ae5 admin admin] PATH_INFO: `/v3/projects/a78444d6606140808d457fc6016729e6/users/77b0f7f6364445058d061e28b16f9f4a/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:58.185509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fc79e-53a7-462e-9ec2-bb0af2899ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78444d6606140808d457fc6016729e6, user_id=77b0f7f6364445058d061e28b16f9f4a, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:58.188231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fc79e-53a7-462e-9ec2-bb0af2899ae5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:58.198330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 935/1889] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:20:58 2022] PUT /identity/v3/projects/a78444d6606140808d457fc6016729e6/users/77b0f7f6364445058d061e28b16f9f4a/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:20:58.208613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef1ca5-818b-416d-a2ba-9058ac23e71d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:58.208853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef1ca5-818b-416d-a2ba-9058ac23e71d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:58.209131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef1ca5-818b-416d-a2ba-9058ac23e71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:58.221052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:58.235597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:58.236568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-52ef1ca5-818b-416d-a2ba-9058ac23e71d None None] MFA Rules not processed for user `77b0f7f6364445058d061e28b16f9f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:20:58.237059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:58.237869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-52ef1ca5-818b-416d-a2ba-9058ac23e71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:58.246561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 955/1890] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:58.318994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 936/1891] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:20:58 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:20:58.329697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:58.331225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:20:58.341964 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff392bc-2c3e-4377-8430-ad7e5268c41a service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:58.391674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708f862-008d-4c5a-8096-484f2246f8a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:58.392361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f862-008d-4c5a-8096-484f2246f8a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:58.392624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f862-008d-4c5a-8096-484f2246f8a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:58.392898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f862-008d-4c5a-8096-484f2246f8a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:58.402320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708f862-008d-4c5a-8096-484f2246f8a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:58.403852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708f862-008d-4c5a-8096-484f2246f8a0 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:58.413273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 956/1892] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:58 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:20:58.639048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef1ca5-818b-416d-a2ba-9058ac23e71d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:20:58.648998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef1ca5-818b-416d-a2ba-9058ac23e71d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:20:58.652830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be9f88-46fd-4043-a6a7-d0eaae2753b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:20:58.653572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be9f88-46fd-4043-a6a7-d0eaae2753b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:20:58.653893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be9f88-46fd-4043-a6a7-d0eaae2753b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:20:58.654231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be9f88-46fd-4043-a6a7-d0eaae2753b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:20:58.704039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:20:58.720601 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be9f88-46fd-4043-a6a7-d0eaae2753b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:20:58.722088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be9f88-46fd-4043-a6a7-d0eaae2753b0 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:20:58.731619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 937/1893] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:20:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:02.468644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708f862-008d-4c5a-8096-484f2246f8a0 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:02.568146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708f862-008d-4c5a-8096-484f2246f8a0 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:02.572653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:02.573513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:02.573793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:02.574071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] PATH_INFO: `/v3/users/d3b55c5c38f24a1c9533e1879e9497c1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:02.575307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b55c5c38f24a1c9533e1879e9497c1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:02.577729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:02.644407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b55c5c38f24a1c9533e1879e9497c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:02.645257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b55c5c38f24a1c9533e1879e9497c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:02.655417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b55c5c38f24a1c9533e1879e9497c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:02.667782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b55c5c38f24a1c9533e1879e9497c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:02.676477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b55c5c38f24a1c9533e1879e9497c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:02.682538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 957/1894] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:02 2022] DELETE /identity/v3/users/d3b55c5c38f24a1c9533e1879e9497c1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:02.690320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399296-7e6b-454b-8352-3237fa1b0b13 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:02.690820 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399296-7e6b-454b-8352-3237fa1b0b13 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:02.691010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399296-7e6b-454b-8352-3237fa1b0b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:02.709705 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7c399296-7e6b-454b-8352-3237fa1b0b13 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:02.710917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7c399296-7e6b-454b-8352-3237fa1b0b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:02.775863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 938/1895] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:02.785932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:02.787595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:02.798802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:02.990428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e03dc-50cf-4a8a-be3f-5b342f7911f6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:02.995078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e521a3b-3aa1-48d9-9591-5eeb7b03ace8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:02.995770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e521a3b-3aa1-48d9-9591-5eeb7b03ace8 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:02.996082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e521a3b-3aa1-48d9-9591-5eeb7b03ace8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:02.996351 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e521a3b-3aa1-48d9-9591-5eeb7b03ace8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:03.006886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e521a3b-3aa1-48d9-9591-5eeb7b03ace8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:03.008406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e521a3b-3aa1-48d9-9591-5eeb7b03ace8 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:03.018131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 958/1896] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:03.299957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c399296-7e6b-454b-8352-3237fa1b0b13 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:03.302157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7c399296-7e6b-454b-8352-3237fa1b0b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:03.321025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c399296-7e6b-454b-8352-3237fa1b0b13 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:03.327116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:03.328134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:03.328463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:03.328834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] PATH_INFO: `/v3/projects/67b5cf2365d54485b7d5cd648ec83520` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:03.331751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b5cf2365d54485b7d5cd648ec83520)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:03.333813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:03.389516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b5cf2365d54485b7d5cd648ec83520', 'request_id': 'req-35ac7140-38a7-4c1d-a608-52b8b7d25696'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.400155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b5cf2365d54485b7d5cd648ec83520', 'request_id': 'req-35ac7140-38a7-4c1d-a608-52b8b7d25696'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.409753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b5cf2365d54485b7d5cd648ec83520', 'request_id': 'req-35ac7140-38a7-4c1d-a608-52b8b7d25696'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.424317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] The token cache is being invalidate because project 67b5cf2365d54485b7d5cd648ec83520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:03.424612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.425275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.427171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 939/1897] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:03 2022] DELETE /identity/v3/projects/67b5cf2365d54485b7d5cd648ec83520 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:03.430975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e521a3b-3aa1-48d9-9591-5eeb7b03ace8 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:03.441572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e521a3b-3aa1-48d9-9591-5eeb7b03ace8 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:03.504430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:03.505325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:03.505611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:03.505884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] PATH_INFO: `/v3/users/92540a87b643498fb5845eda53d6b2f6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:03.507077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=92540a87b643498fb5845eda53d6b2f6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:03.509032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:03.557622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92540a87b643498fb5845eda53d6b2f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.558271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92540a87b643498fb5845eda53d6b2f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.565385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92540a87b643498fb5845eda53d6b2f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.577029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92540a87b643498fb5845eda53d6b2f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.585350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92540a87b643498fb5845eda53d6b2f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:03.591287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 959/1898] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:03 2022] DELETE /identity/v3/users/92540a87b643498fb5845eda53d6b2f6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:04.052738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:04.054350 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:04.068069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac7140-38a7-4c1d-a608-52b8b7d25696 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:04.132500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:04.133415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:04.133719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:04.133997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] PATH_INFO: `/v3/projects/74e802ca024e4f46889b17288c6ed79c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:04.135235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e802ca024e4f46889b17288c6ed79c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:04.137221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:04.148687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:04.187465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e802ca024e4f46889b17288c6ed79c', 'request_id': 'req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.192944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e802ca024e4f46889b17288c6ed79c', 'request_id': 'req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.201804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e802ca024e4f46889b17288c6ed79c', 'request_id': 'req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.213523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] The token cache is being invalidate because project 74e802ca024e4f46889b17288c6ed79c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:04.213816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.214542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.216465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 940/1899] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:04 2022] DELETE /identity/v3/projects/74e802ca024e4f46889b17288c6ed79c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:04.220659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:04.222257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:04.235169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440c63-3406-4a3f-a021-f5f3d50e1ead admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:04.261545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:04.298224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:04.299038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:04.299342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:04.299615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] PATH_INFO: `/v3/users/bd307c04c65a469e9f34d4fc995e31c9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:04.300946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd307c04c65a469e9f34d4fc995e31c9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:04.302908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:04.356139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd307c04c65a469e9f34d4fc995e31c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.356794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd307c04c65a469e9f34d4fc995e31c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.364312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd307c04c65a469e9f34d4fc995e31c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.377664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd307c04c65a469e9f34d4fc995e31c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.387170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4c636f96-bad8-4f9b-b635-ef9cef49aa5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd307c04c65a469e9f34d4fc995e31c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.393971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 960/1900] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:04 2022] DELETE /identity/v3/users/bd307c04c65a469e9f34d4fc995e31c9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:04.680378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:04.681976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:04.695548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a5d6e-085e-4c0a-bfb5-c81bffc294ae admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:04.752821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:04.753717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:04.754001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:04.754280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] PATH_INFO: `/v3/projects/11fd062d57a34558bb4690516fb9c53a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:04.755501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fd062d57a34558bb4690516fb9c53a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:04.757481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:04.816856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fd062d57a34558bb4690516fb9c53a', 'request_id': 'req-598d20b8-9bff-481e-a930-8b59d1530e7d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.822402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fd062d57a34558bb4690516fb9c53a', 'request_id': 'req-598d20b8-9bff-481e-a930-8b59d1530e7d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.832625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fd062d57a34558bb4690516fb9c53a', 'request_id': 'req-598d20b8-9bff-481e-a930-8b59d1530e7d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.848890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] The token cache is being invalidate because project 11fd062d57a34558bb4690516fb9c53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:04.849180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.849965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-598d20b8-9bff-481e-a930-8b59d1530e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:04.851763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 941/1901] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:04 2022] DELETE /identity/v3/projects/11fd062d57a34558bb4690516fb9c53a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:04.858017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2dd0-d338-4ac1-8eb9-4484786f68de None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:04.859045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2dd0-d338-4ac1-8eb9-4484786f68de None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:04.859045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2dd0-d338-4ac1-8eb9-4484786f68de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:04.883628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dabe2dd0-d338-4ac1-8eb9-4484786f68de None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:04.884847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dabe2dd0-d338-4ac1-8eb9-4484786f68de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:04.949186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 961/1902] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:04.957502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be89c0-13e1-4172-9a96-181fcd4d696d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:04.958408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be89c0-13e1-4172-9a96-181fcd4d696d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:04.958408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be89c0-13e1-4172-9a96-181fcd4d696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:04.976999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-91be89c0-13e1-4172-9a96-181fcd4d696d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:04.978232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-91be89c0-13e1-4172-9a96-181fcd4d696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:04.986783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 942/1903] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:04.992329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2dd0-d338-4ac1-8eb9-4484786f68de None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.002730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2dd0-d338-4ac1-8eb9-4484786f68de None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.008604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.009468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.009668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.010479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.010902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:21:05.011334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.013103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.026250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 962/1904] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:21:05.030909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be89c0-13e1-4172-9a96-181fcd4d696d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.040198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be89c0-13e1-4172-9a96-181fcd4d696d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.045328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cde8f7-4b37-477b-9c58-116c7bb64ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.046137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cde8f7-4b37-477b-9c58-116c7bb64ebf admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.046449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cde8f7-4b37-477b-9c58-116c7bb64ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.046723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cde8f7-4b37-477b-9c58-116c7bb64ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.047455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cde8f7-4b37-477b-9c58-116c7bb64ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.049359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cde8f7-4b37-477b-9c58-116c7bb64ebf admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.079126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 943/1905] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:05 2022] POST /identity/v3/projects => generated 359 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:05.080088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.090000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6db1c-d112-4531-a919-0c9a85881871 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.122665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:05.151944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a15f6-f272-4a13-abde-4a679c658771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.152664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a15f6-f272-4a13-abde-4a679c658771 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.152950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a15f6-f272-4a13-abde-4a679c658771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.153221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a15f6-f272-4a13-abde-4a679c658771 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.154065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a15f6-f272-4a13-abde-4a679c658771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.156072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a15f6-f272-4a13-abde-4a679c658771 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.180126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 963/1906] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:05 2022] POST /identity/v3/users => generated 280 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:05.181220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cde8f7-4b37-477b-9c58-116c7bb64ebf admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.186909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:05.192137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cde8f7-4b37-477b-9c58-116c7bb64ebf admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.198701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac37fb-329e-4041-b72c-e0fbb76077ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.199806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac37fb-329e-4041-b72c-e0fbb76077ec admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.200112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac37fb-329e-4041-b72c-e0fbb76077ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.201009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac37fb-329e-4041-b72c-e0fbb76077ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.202201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac37fb-329e-4041-b72c-e0fbb76077ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.204191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac37fb-329e-4041-b72c-e0fbb76077ec admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.216193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 944/1907] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:05.220500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a15f6-f272-4a13-abde-4a679c658771 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.231764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a15f6-f272-4a13-abde-4a679c658771 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.236480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1bda-2985-4193-ba8a-5f090bf4c4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.237154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1bda-2985-4193-ba8a-5f090bf4c4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.237442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1bda-2985-4193-ba8a-5f090bf4c4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.237722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1bda-2985-4193-ba8a-5f090bf4c4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.238457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd1bda-2985-4193-ba8a-5f090bf4c4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.240132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd1bda-2985-4193-ba8a-5f090bf4c4c1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.250718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 964/1908] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:05.254929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac37fb-329e-4041-b72c-e0fbb76077ec admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.267573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac37fb-329e-4041-b72c-e0fbb76077ec admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.272695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.273607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.273923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.274188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] PATH_INFO: `/v3/projects/a65b1be71c8443cbb17d07a228a046c1/users/7f7c206a95f34a59ae8634956b352082/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.290942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65b1be71c8443cbb17d07a228a046c1, user_id=7f7c206a95f34a59ae8634956b352082, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.293618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.308144 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 945/1909] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:05 2022] PUT /identity/v3/projects/a65b1be71c8443cbb17d07a228a046c1/users/7f7c206a95f34a59ae8634956b352082/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:05.319660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f9158-24fb-4617-8ff6-4ea1c6823ae0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.319950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f9158-24fb-4617-8ff6-4ea1c6823ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.320231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f9158-24fb-4617-8ff6-4ea1c6823ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.349484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-837f9158-24fb-4617-8ff6-4ea1c6823ae0 None None] MFA Rules not processed for user `7f7c206a95f34a59ae8634956b352082`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:05.350675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-837f9158-24fb-4617-8ff6-4ea1c6823ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:05.423723 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 965/1910] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:05 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:05.459048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.460958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:05.534493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608e0b1-6fec-45e9-a1a2-8ee826becee9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.539763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c79fed-e5c7-46c7-9fb0-c3b5706504ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.540662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c79fed-e5c7-46c7-9fb0-c3b5706504ef service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.540970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c79fed-e5c7-46c7-9fb0-c3b5706504ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.541238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c79fed-e5c7-46c7-9fb0-c3b5706504ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.551804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c79fed-e5c7-46c7-9fb0-c3b5706504ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.553250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c79fed-e5c7-46c7-9fb0-c3b5706504ef service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.563007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 946/1911] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:05 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:05.576798 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f9158-24fb-4617-8ff6-4ea1c6823ae0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.644064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f9158-24fb-4617-8ff6-4ea1c6823ae0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.648702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.649596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.649873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.650135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] PATH_INFO: `/v3/users/7f7c206a95f34a59ae8634956b352082` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.651353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7c206a95f34a59ae8634956b352082)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.653275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.702539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:05.704667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7c206a95f34a59ae8634956b352082'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:05.707629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7c206a95f34a59ae8634956b352082'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:05.712280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890225dc-2155-48ab-9ab5-909426c09e87 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.713517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890225dc-2155-48ab-9ab5-909426c09e87 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.714383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890225dc-2155-48ab-9ab5-909426c09e87 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.717532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 947/1912] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:21:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 08:21:05.718045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7c206a95f34a59ae8634956b352082'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:05.725087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626af17-41bf-470f-88ef-f50dc8cbe062 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.725533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626af17-41bf-470f-88ef-f50dc8cbe062 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.725909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626af17-41bf-470f-88ef-f50dc8cbe062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.743935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7c206a95f34a59ae8634956b352082'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:05.748937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3626af17-41bf-470f-88ef-f50dc8cbe062 None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:05.750788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3626af17-41bf-470f-88ef-f50dc8cbe062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:05.753679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-014d69ac-8dc6-4cc0-8dad-15fc2d57e922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7c206a95f34a59ae8634956b352082'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:05.764411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 966/1913] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:05 2022] DELETE /identity/v3/users/7f7c206a95f34a59ae8634956b352082 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:05.766110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112e7ff-1baa-48cc-9f74-e5a411604a46 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.766580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112e7ff-1baa-48cc-9f74-e5a411604a46 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.766912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112e7ff-1baa-48cc-9f74-e5a411604a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.794615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f112e7ff-1baa-48cc-9f74-e5a411604a46 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:05.796093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f112e7ff-1baa-48cc-9f74-e5a411604a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:05.832944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 948/1914] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:21:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:05.848291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626af17-41bf-470f-88ef-f50dc8cbe062 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.850163 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3626af17-41bf-470f-88ef-f50dc8cbe062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:05.863520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 967/1915] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:05.874941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626af17-41bf-470f-88ef-f50dc8cbe062 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.875667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112e7ff-1baa-48cc-9f74-e5a411604a46 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:05.877336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f112e7ff-1baa-48cc-9f74-e5a411604a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:05.932552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:05.941457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112e7ff-1baa-48cc-9f74-e5a411604a46 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:05.945316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.946109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.946400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.946733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.947612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4205ff-4bed-40b0-ba36-8fef661b310b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:05.948378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4205ff-4bed-40b0-ba36-8fef661b310b service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:05.948754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4205ff-4bed-40b0-ba36-8fef661b310b service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:05.949264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4205ff-4bed-40b0-ba36-8fef661b310b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:05.957804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.959495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.959643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4205ff-4bed-40b0-ba36-8fef661b310b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:05.961307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4205ff-4bed-40b0-ba36-8fef661b310b service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:05.970918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 968/1916] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:21:05.974507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 949/1917] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:21:06.367414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:06.369111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:06.384002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874c4c-0aee-4893-b3a9-ef55c60a2564 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:06.389191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:06.390015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.390294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.390564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] PATH_INFO: `/v3/projects/a65b1be71c8443cbb17d07a228a046c1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:06.391770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65b1be71c8443cbb17d07a228a046c1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:06.393902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:06.455462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65b1be71c8443cbb17d07a228a046c1', 'request_id': 'req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:06.461741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65b1be71c8443cbb17d07a228a046c1', 'request_id': 'req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:06.464747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:06.472844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65b1be71c8443cbb17d07a228a046c1', 'request_id': 'req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:06.495138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] The token cache is being invalidate because project a65b1be71c8443cbb17d07a228a046c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:06.495541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:06.496192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0f18508-45d1-491f-9fa0-88b3b6eab4d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:06.497858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 969/1918] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:06 2022] DELETE /identity/v3/projects/a65b1be71c8443cbb17d07a228a046c1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:06.503170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c913ad-83e6-42dc-8092-112ee71599bc None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.503638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c913ad-83e6-42dc-8092-112ee71599bc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.503804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c913ad-83e6-42dc-8092-112ee71599bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:06.522055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a7c913ad-83e6-42dc-8092-112ee71599bc None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:06.523303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a7c913ad-83e6-42dc-8092-112ee71599bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:06.589842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 950/1919] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:06.600376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56fa4-a073-4b9d-9265-07fecc14155f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.600668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56fa4-a073-4b9d-9265-07fecc14155f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.600974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56fa4-a073-4b9d-9265-07fecc14155f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:06.621888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-10f56fa4-a073-4b9d-9265-07fecc14155f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:06.623671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10f56fa4-a073-4b9d-9265-07fecc14155f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:06.640137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 970/1920] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:06.644426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c913ad-83e6-42dc-8092-112ee71599bc None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:06.654970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c913ad-83e6-42dc-8092-112ee71599bc None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:06.660612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:06.661318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.661602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.661879 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:06.662766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:21:06.663086 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:06.664785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:06.681446 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 951/1921] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:21:06.682812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f56fa4-a073-4b9d-9265-07fecc14155f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:06.692957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f56fa4-a073-4b9d-9265-07fecc14155f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:06.700325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f232f-5cb6-43b2-bd72-bcd969ddd2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:06.701367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f232f-5cb6-43b2-bd72-bcd969ddd2bb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.701861 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f232f-5cb6-43b2-bd72-bcd969ddd2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.701998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f232f-5cb6-43b2-bd72-bcd969ddd2bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:06.702634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f232f-5cb6-43b2-bd72-bcd969ddd2bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:06.704888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f232f-5cb6-43b2-bd72-bcd969ddd2bb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:06.740065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 971/1922] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:06 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:06.741251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:06.752787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86835-49dc-4f11-87c7-4d2a7cd53aa0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:06.824517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ca65e-4b48-49db-afda-80020b5206fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:06.825454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca65e-4b48-49db-afda-80020b5206fe admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.826137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca65e-4b48-49db-afda-80020b5206fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.826571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca65e-4b48-49db-afda-80020b5206fe admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:06.827491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ca65e-4b48-49db-afda-80020b5206fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:06.829624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ca65e-4b48-49db-afda-80020b5206fe admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:06.852951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 952/1923] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:06 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:06.857527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f232f-5cb6-43b2-bd72-bcd969ddd2bb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:06.868393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f232f-5cb6-43b2-bd72-bcd969ddd2bb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:06.877173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b4fd0-6a46-491e-87d6-2583925c63f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:06.878741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b4fd0-6a46-491e-87d6-2583925c63f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.879191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b4fd0-6a46-491e-87d6-2583925c63f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.879493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b4fd0-6a46-491e-87d6-2583925c63f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:06.881988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b4fd0-6a46-491e-87d6-2583925c63f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:06.884725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b4fd0-6a46-491e-87d6-2583925c63f2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:06.901302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 972/1924] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:06.905643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ca65e-4b48-49db-afda-80020b5206fe admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:06.920476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ca65e-4b48-49db-afda-80020b5206fe admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:06.928083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ebdf-5f10-445e-bdc1-47b763d52995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:06.928899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ebdf-5f10-445e-bdc1-47b763d52995 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.929291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ebdf-5f10-445e-bdc1-47b763d52995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.929758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ebdf-5f10-445e-bdc1-47b763d52995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:06.930981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6ebdf-5f10-445e-bdc1-47b763d52995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:06.933018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6ebdf-5f10-445e-bdc1-47b763d52995 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:06.949804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 953/1925] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:06.955006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b4fd0-6a46-491e-87d6-2583925c63f2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:06.971891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b4fd0-6a46-491e-87d6-2583925c63f2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:06.977405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe1580-3dac-4f78-81cb-125729f7b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:06.980074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe1580-3dac-4f78-81cb-125729f7b333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:06.980506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe1580-3dac-4f78-81cb-125729f7b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:06.981077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe1580-3dac-4f78-81cb-125729f7b333 admin admin] PATH_INFO: `/v3/projects/5f83dba68b0d4d2c90a91e950ac9b183/users/bd3e7d31890b432ab2fb772501557c2e/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.005916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe1580-3dac-4f78-81cb-125729f7b333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f83dba68b0d4d2c90a91e950ac9b183, user_id=bd3e7d31890b432ab2fb772501557c2e, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.008709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe1580-3dac-4f78-81cb-125729f7b333 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.024491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 973/1926] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:06 2022] PUT /identity/v3/projects/5f83dba68b0d4d2c90a91e950ac9b183/users/bd3e7d31890b432ab2fb772501557c2e/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:07.038785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e41510-92c0-45f7-b94c-e8b032bd40d3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.039333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e41510-92c0-45f7-b94c-e8b032bd40d3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.039674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e41510-92c0-45f7-b94c-e8b032bd40d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.075134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-61e41510-92c0-45f7-b94c-e8b032bd40d3 None None] MFA Rules not processed for user `bd3e7d31890b432ab2fb772501557c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:07.076741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-61e41510-92c0-45f7-b94c-e8b032bd40d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:07.121098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:07.145958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 954/1927] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:07.150540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe1580-3dac-4f78-81cb-125729f7b333 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.214401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe1580-3dac-4f78-81cb-125729f7b333 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.219000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acf224-9aec-4608-bc0b-674bf1f1e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.219698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf224-9aec-4608-bc0b-674bf1f1e070 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.219979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf224-9aec-4608-bc0b-674bf1f1e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.220250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf224-9aec-4608-bc0b-674bf1f1e070 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.220951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acf224-9aec-4608-bc0b-674bf1f1e070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.222872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acf224-9aec-4608-bc0b-674bf1f1e070 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.248840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 974/1928] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:07 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:07.252962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e41510-92c0-45f7-b94c-e8b032bd40d3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.265356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e41510-92c0-45f7-b94c-e8b032bd40d3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.342972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.343874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.344208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.344648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.345443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.347509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.375027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 955/1929] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:07 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:07.379222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acf224-9aec-4608-bc0b-674bf1f1e070 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.394082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acf224-9aec-4608-bc0b-674bf1f1e070 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.399935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e98caa-a8d3-472b-95b8-b514e8840bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.400922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e98caa-a8d3-472b-95b8-b514e8840bbb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.401377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e98caa-a8d3-472b-95b8-b514e8840bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.401677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e98caa-a8d3-472b-95b8-b514e8840bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.402923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e98caa-a8d3-472b-95b8-b514e8840bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.404937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e98caa-a8d3-472b-95b8-b514e8840bbb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.406445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.408607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:07.427315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d257eb6-ac55-45cc-806e-d4cf6f0347b9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.432390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.432532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 975/1930] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:07.433624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.434037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.434751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] PATH_INFO: `/v3/users/9d2fb7f24bf8495e937a1c353cc3cf83` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.437183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e98caa-a8d3-472b-95b8-b514e8840bbb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.437322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2fb7f24bf8495e937a1c353cc3cf83)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.439719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.449005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e98caa-a8d3-472b-95b8-b514e8840bbb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.455690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bbd8-8c5e-4679-b3f7-75d6259e5285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.456370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bbd8-8c5e-4679-b3f7-75d6259e5285 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.456653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bbd8-8c5e-4679-b3f7-75d6259e5285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.456920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bbd8-8c5e-4679-b3f7-75d6259e5285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.457893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012bbd8-8c5e-4679-b3f7-75d6259e5285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.459376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012bbd8-8c5e-4679-b3f7-75d6259e5285 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.476240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 976/1931] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:07.481404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bbd8-8c5e-4679-b3f7-75d6259e5285 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.512585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bbd8-8c5e-4679-b3f7-75d6259e5285 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.524859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c03534-9098-467b-9b75-7d43df8dc91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.525979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03534-9098-467b-9b75-7d43df8dc91d admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.526378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03534-9098-467b-9b75-7d43df8dc91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.526749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03534-9098-467b-9b75-7d43df8dc91d admin admin] PATH_INFO: `/v3/projects/0ea26f9337134a25ae8cde6100ebd409/users/db7ceb0af18549358f270780f74ec717/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.530045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2fb7f24bf8495e937a1c353cc3cf83'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:07.539646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2fb7f24bf8495e937a1c353cc3cf83'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:07.548574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2fb7f24bf8495e937a1c353cc3cf83'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:07.549521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2fb7f24bf8495e937a1c353cc3cf83'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:07.556912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c03534-9098-467b-9b75-7d43df8dc91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea26f9337134a25ae8cde6100ebd409, user_id=db7ceb0af18549358f270780f74ec717, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.567094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c03534-9098-467b-9b75-7d43df8dc91d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.571909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ae8e2e5-532b-4163-87b0-60578eb35d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2fb7f24bf8495e937a1c353cc3cf83'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:07.585190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 956/1932] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:07 2022] DELETE /identity/v3/users/9d2fb7f24bf8495e937a1c353cc3cf83 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:07.590112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 977/1933] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:07 2022] PUT /identity/v3/projects/0ea26f9337134a25ae8cde6100ebd409/users/db7ceb0af18549358f270780f74ec717/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:07.590895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48125509-9798-46c0-9b09-18dbfcf977b6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.591407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48125509-9798-46c0-9b09-18dbfcf977b6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.591980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48125509-9798-46c0-9b09-18dbfcf977b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.600465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dc5aa-c95f-4798-8e27-cb4159d434ee None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.601247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dc5aa-c95f-4798-8e27-cb4159d434ee None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.601609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dc5aa-c95f-4798-8e27-cb4159d434ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.628680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-48125509-9798-46c0-9b09-18dbfcf977b6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:07.630216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-48125509-9798-46c0-9b09-18dbfcf977b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:07.635755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-563dc5aa-c95f-4798-8e27-cb4159d434ee None None] MFA Rules not processed for user `db7ceb0af18549358f270780f74ec717`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:07.637335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-563dc5aa-c95f-4798-8e27-cb4159d434ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:07.652093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:07.680131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:07.714265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 957/1934] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:21:07.724240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48125509-9798-46c0-9b09-18dbfcf977b6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.726186 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-48125509-9798-46c0-9b09-18dbfcf977b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:07.738780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 978/1935] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:21:07.740036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dc5aa-c95f-4798-8e27-cb4159d434ee None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.741439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-563dc5aa-c95f-4798-8e27-cb4159d434ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:07.756349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dc5aa-c95f-4798-8e27-cb4159d434ee None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.761435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187fcb9-ed12-41fc-823d-86039146ba6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.763141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187fcb9-ed12-41fc-823d-86039146ba6f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.763610 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187fcb9-ed12-41fc-823d-86039146ba6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.763891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187fcb9-ed12-41fc-823d-86039146ba6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.764772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187fcb9-ed12-41fc-823d-86039146ba6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.767082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187fcb9-ed12-41fc-823d-86039146ba6f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.790927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48125509-9798-46c0-9b09-18dbfcf977b6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.794873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a7b00-1806-46a8-ad1d-768eebb464e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.795555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7b00-1806-46a8-ad1d-768eebb464e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.795834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7b00-1806-46a8-ad1d-768eebb464e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.796106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7b00-1806-46a8-ad1d-768eebb464e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.799690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 979/1936] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:07 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:07.800781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187fcb9-ed12-41fc-823d-86039146ba6f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.816221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a7b00-1806-46a8-ad1d-768eebb464e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.817923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a7b00-1806-46a8-ad1d-768eebb464e3 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.830573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 958/1937] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:21:07.873628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187fcb9-ed12-41fc-823d-86039146ba6f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.883117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35319a0c-7133-41b0-8d80-8e3b397cb804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.886334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35319a0c-7133-41b0-8d80-8e3b397cb804 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.886742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35319a0c-7133-41b0-8d80-8e3b397cb804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.887068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35319a0c-7133-41b0-8d80-8e3b397cb804 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.887907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35319a0c-7133-41b0-8d80-8e3b397cb804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.890316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35319a0c-7133-41b0-8d80-8e3b397cb804 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.917334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 980/1938] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:07 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:07.921041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a7b00-1806-46a8-ad1d-768eebb464e3 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.932176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a7b00-1806-46a8-ad1d-768eebb464e3 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.937574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ce05e-9720-4213-aa09-d20dce620cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.938498 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ce05e-9720-4213-aa09-d20dce620cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.938780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ce05e-9720-4213-aa09-d20dce620cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.939136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ce05e-9720-4213-aa09-d20dce620cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.940260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ce05e-9720-4213-aa09-d20dce620cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.942220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ce05e-9720-4213-aa09-d20dce620cc0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:07.954824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 959/1939] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:21:07.958912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35319a0c-7133-41b0-8d80-8e3b397cb804 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:07.969446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35319a0c-7133-41b0-8d80-8e3b397cb804 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:07.974163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd98ae-76a4-49d8-870f-f346828a1d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:07.975129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd98ae-76a4-49d8-870f-f346828a1d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:07.975354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd98ae-76a4-49d8-870f-f346828a1d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:07.975625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd98ae-76a4-49d8-870f-f346828a1d16 admin admin] PATH_INFO: `/v3/projects/81cdabe37b264181aeb1f18cf11a4551/users/b388cb41f2e0408088bb8ee405619515/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:07.991599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd98ae-76a4-49d8-870f-f346828a1d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81cdabe37b264181aeb1f18cf11a4551, user_id=b388cb41f2e0408088bb8ee405619515, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:07.994427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd98ae-76a4-49d8-870f-f346828a1d16 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.011499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 981/1940] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:07 2022] PUT /identity/v3/projects/81cdabe37b264181aeb1f18cf11a4551/users/b388cb41f2e0408088bb8ee405619515/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:08.015857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ce05e-9720-4213-aa09-d20dce620cc0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.026516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ce05e-9720-4213-aa09-d20dce620cc0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.115425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.115425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.115425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.115425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.115938 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.117608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.131744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 960/1941] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:08.135658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd98ae-76a4-49d8-870f-f346828a1d16 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.162055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd98ae-76a4-49d8-870f-f346828a1d16 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.175628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.177449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:08.188742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad21645-192d-4744-a041-807055e63336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.189664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad21645-192d-4744-a041-807055e63336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.190022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad21645-192d-4744-a041-807055e63336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.190196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad21645-192d-4744-a041-807055e63336 admin admin] PATH_INFO: `/v3/projects/81cdabe37b264181aeb1f18cf11a4551/users/b388cb41f2e0408088bb8ee405619515/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.195827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad21645-192d-4744-a041-807055e63336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81cdabe37b264181aeb1f18cf11a4551, user_id=b388cb41f2e0408088bb8ee405619515, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.198520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad21645-192d-4744-a041-807055e63336 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.201640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c226f20-7232-46ae-b7b5-7a0e7b330ff8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.215222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.216692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.217042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.217352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] PATH_INFO: `/v3/projects/3722dba444a14413a0c8e3155a56e437` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.221017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3722dba444a14413a0c8e3155a56e437)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.226920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.229808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 982/1942] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:08 2022] PUT /identity/v3/projects/81cdabe37b264181aeb1f18cf11a4551/users/b388cb41f2e0408088bb8ee405619515/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:08.242387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee4223-2bc7-4ef3-bc5c-438db35b7cb3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.242708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee4223-2bc7-4ef3-bc5c-438db35b7cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.242992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee4223-2bc7-4ef3-bc5c-438db35b7cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.293663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ddee4223-2bc7-4ef3-bc5c-438db35b7cb3 None None] MFA Rules not processed for user `b388cb41f2e0408088bb8ee405619515`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:08.295249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ddee4223-2bc7-4ef3-bc5c-438db35b7cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:08.325898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3722dba444a14413a0c8e3155a56e437', 'request_id': 'req-0818229b-051b-4b4b-9895-8ce45e6a2b0a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:08.337028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3722dba444a14413a0c8e3155a56e437', 'request_id': 'req-0818229b-051b-4b4b-9895-8ce45e6a2b0a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:08.346030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3722dba444a14413a0c8e3155a56e437', 'request_id': 'req-0818229b-051b-4b4b-9895-8ce45e6a2b0a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:08.350184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:08.351378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:08.381260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] The token cache is being invalidate because project 3722dba444a14413a0c8e3155a56e437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:08.381743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:08.383746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0818229b-051b-4b4b-9895-8ce45e6a2b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:08.386158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 961/1943] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:08 2022] DELETE /identity/v3/projects/3722dba444a14413a0c8e3155a56e437 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:08.391285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263128e-294d-429d-9276-a59a7f267cae None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.392466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263128e-294d-429d-9276-a59a7f267cae None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.393001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263128e-294d-429d-9276-a59a7f267cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.408320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 983/1944] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:21:08.421912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4263128e-294d-429d-9276-a59a7f267cae None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:08.423310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4263128e-294d-429d-9276-a59a7f267cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:08.424423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee4223-2bc7-4ef3-bc5c-438db35b7cb3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.425412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ddee4223-2bc7-4ef3-bc5c-438db35b7cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:08.496297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee4223-2bc7-4ef3-bc5c-438db35b7cb3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.502223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.502781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 962/1945] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:21:08.503092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.503405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.503729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.509990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:08.523045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1beb11-c626-4f8d-b058-b2d6c378e391 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.531781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1beb11-c626-4f8d-b058-b2d6c378e391 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.532735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1beb11-c626-4f8d-b058-b2d6c378e391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.559806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8c1beb11-c626-4f8d-b058-b2d6c378e391 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:08.561724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8c1beb11-c626-4f8d-b058-b2d6c378e391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:08.573571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 963/1946] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:08.574784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1beb11-c626-4f8d-b058-b2d6c378e391 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.587202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1beb11-c626-4f8d-b058-b2d6c378e391 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.592822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.593702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.593987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.594386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.595129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:21:08.595475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.595588 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.597227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.597762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.611326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 964/1947] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:21:08.613515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 984/1948] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:21:08.615912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.630344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f22fb-4a03-4a0e-a454-cb00bd4c2624 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.646368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb0f6c-737a-4e01-8b08-f6fea0052f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.647142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0f6c-737a-4e01-8b08-f6fea0052f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.647537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0f6c-737a-4e01-8b08-f6fea0052f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.647889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0f6c-737a-4e01-8b08-f6fea0052f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.648684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb0f6c-737a-4e01-8b08-f6fea0052f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.650743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb0f6c-737a-4e01-8b08-f6fea0052f38 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.682907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 965/1949] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:08 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:08.688314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.705885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c1cba-06d5-4a81-a926-a89882e0acd0 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.790116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93c4c7-4b2c-4e16-beeb-bbceb3166a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.790699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93c4c7-4b2c-4e16-beeb-bbceb3166a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.790988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93c4c7-4b2c-4e16-beeb-bbceb3166a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.791311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93c4c7-4b2c-4e16-beeb-bbceb3166a99 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.792017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93c4c7-4b2c-4e16-beeb-bbceb3166a99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.794540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93c4c7-4b2c-4e16-beeb-bbceb3166a99 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.818959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 985/1950] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:08 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:08.823406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb0f6c-737a-4e01-8b08-f6fea0052f38 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.836720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb0f6c-737a-4e01-8b08-f6fea0052f38 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.842830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c85844-a3ab-448f-875c-1c30dce644db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.843554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c85844-a3ab-448f-875c-1c30dce644db admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.843834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c85844-a3ab-448f-875c-1c30dce644db admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.844235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c85844-a3ab-448f-875c-1c30dce644db admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.844980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c85844-a3ab-448f-875c-1c30dce644db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.846693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c85844-a3ab-448f-875c-1c30dce644db admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.857888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 966/1951] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:21:08.863003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93c4c7-4b2c-4e16-beeb-bbceb3166a99 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.875349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93c4c7-4b2c-4e16-beeb-bbceb3166a99 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.885211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177e5eb-a574-4c30-b28b-c10f06913ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.886477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e5eb-a574-4c30-b28b-c10f06913ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.886477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e5eb-a574-4c30-b28b-c10f06913ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.886776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e5eb-a574-4c30-b28b-c10f06913ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.887882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177e5eb-a574-4c30-b28b-c10f06913ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.890048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177e5eb-a574-4c30-b28b-c10f06913ef7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.905957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 986/1952] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:08.907416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c85844-a3ab-448f-875c-1c30dce644db admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:08.926960 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c85844-a3ab-448f-875c-1c30dce644db admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:08.926960 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:08.926960 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.926960 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.928076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] PATH_INFO: `/v3/projects/bdbc517ae372470e87e5e73c1f326798/users/c62a18ec68d5494e912f101ee9b1ebad/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.942491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdbc517ae372470e87e5e73c1f326798, user_id=c62a18ec68d5494e912f101ee9b1ebad, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:08.945166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:08.964807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 967/1953] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:08 2022] PUT /identity/v3/projects/bdbc517ae372470e87e5e73c1f326798/users/c62a18ec68d5494e912f101ee9b1ebad/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:08.976008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467275b-1695-4dee-9706-683a488c52c5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:08.976302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467275b-1695-4dee-9706-683a488c52c5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:08.976579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467275b-1695-4dee-9706-683a488c52c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:08.983699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:09.016969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5467275b-1695-4dee-9706-683a488c52c5 None None] MFA Rules not processed for user `c62a18ec68d5494e912f101ee9b1ebad`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:09.018431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5467275b-1695-4dee-9706-683a488c52c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:09.083946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 987/1954] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:09.095837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:09.097903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:09.159487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33a672-dd96-451b-932c-f6e41896f634 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:09.163506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef9f70-b251-4abf-b0d1-00fa9b684e1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:09.164144 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef9f70-b251-4abf-b0d1-00fa9b684e1a service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:09.164422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef9f70-b251-4abf-b0d1-00fa9b684e1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:09.164694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef9f70-b251-4abf-b0d1-00fa9b684e1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:09.173679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef9f70-b251-4abf-b0d1-00fa9b684e1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:09.175164 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef9f70-b251-4abf-b0d1-00fa9b684e1a service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:09.184293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 968/1955] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:09.586395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467275b-1695-4dee-9706-683a488c52c5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:09.596642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5467275b-1695-4dee-9706-683a488c52c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:09.701759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467275b-1695-4dee-9706-683a488c52c5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:09.708285 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:09.710238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:09.710673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:09.711524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] PATH_INFO: `/v3/users/be04cce95bd94949b752c6ba92c95d8d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:09.713273 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be04cce95bd94949b752c6ba92c95d8d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:09.716221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:09.789138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be04cce95bd94949b752c6ba92c95d8d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:09.790059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be04cce95bd94949b752c6ba92c95d8d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:09.802758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be04cce95bd94949b752c6ba92c95d8d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:09.818593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be04cce95bd94949b752c6ba92c95d8d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:09.830605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be04cce95bd94949b752c6ba92c95d8d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:09.841523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 988/1956] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:09 2022] DELETE /identity/v3/users/be04cce95bd94949b752c6ba92c95d8d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:09.846541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9ffc3-4ca9-4e29-99a8-1a4ea170f917 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:09.847029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9ffc3-4ca9-4e29-99a8-1a4ea170f917 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:09.847953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9ffc3-4ca9-4e29-99a8-1a4ea170f917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:09.876482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-88c9ffc3-4ca9-4e29-99a8-1a4ea170f917 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:09.878125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-88c9ffc3-4ca9-4e29-99a8-1a4ea170f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:09.905903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:09.966110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 969/1957] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:21:09.981196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:09.982896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:10.009819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:10.110669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55003a-ad34-4774-9d2e-7ec1bd023f06 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:10.146677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:10.147404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:10.147692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:10.147951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:10.179862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:10.181360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:10.203633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 989/1958] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:10.340559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9ffc3-4ca9-4e29-99a8-1a4ea170f917 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:10.341955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-88c9ffc3-4ca9-4e29-99a8-1a4ea170f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:10.407841 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9ffc3-4ca9-4e29-99a8-1a4ea170f917 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:10.411986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:10.412560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:10.412845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:10.413118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:10.415002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:10.484658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:10.486122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:10.500348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 970/1959] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:10.543491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:10.545935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:10.571630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c94a1-385b-4b59-8565-8e656e506851 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:10.578056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:10.578979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:10.579293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:10.579571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] PATH_INFO: `/v3/projects/89ae95bbd5aa44a897eec9810c122e7d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:10.580875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ae95bbd5aa44a897eec9810c122e7d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:10.582813 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:10.651346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ae95bbd5aa44a897eec9810c122e7d', 'request_id': 'req-13a51390-9036-4201-86c7-d1f0af9ade78'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.665908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ae95bbd5aa44a897eec9810c122e7d', 'request_id': 'req-13a51390-9036-4201-86c7-d1f0af9ade78'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.671052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ae95bbd5aa44a897eec9810c122e7d', 'request_id': 'req-13a51390-9036-4201-86c7-d1f0af9ade78'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.693545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] The token cache is being invalidate because project 89ae95bbd5aa44a897eec9810c122e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:10.694116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.694839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.700339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 990/1960] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:10 2022] DELETE /identity/v3/projects/89ae95bbd5aa44a897eec9810c122e7d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:10.701898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:10.715290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc103-e575-489f-b4d2-41f44d5374fb service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:10.812256 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:10.813396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:10.813797 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:10.814199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] PATH_INFO: `/v3/users/77b0f7f6364445058d061e28b16f9f4a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:10.815640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b0f7f6364445058d061e28b16f9f4a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:10.817806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:10.854541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:10.896228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b0f7f6364445058d061e28b16f9f4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.906875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b0f7f6364445058d061e28b16f9f4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.916048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b0f7f6364445058d061e28b16f9f4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.916777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b0f7f6364445058d061e28b16f9f4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.936668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b0f7f6364445058d061e28b16f9f4a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:10.944962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 971/1961] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:10 2022] DELETE /identity/v3/users/77b0f7f6364445058d061e28b16f9f4a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:11.211864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:11.213223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:11.301752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51390-9036-4201-86c7-d1f0af9ade78 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:11.351084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:11.352801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:11.382422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1dc66-aee5-4610-98cd-78752bb4df42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:11.383838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1dc66-aee5-4610-98cd-78752bb4df42 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:11.384432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1dc66-aee5-4610-98cd-78752bb4df42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:11.384721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1dc66-aee5-4610-98cd-78752bb4df42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:11.387331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f0d1dc66-aee5-4610-98cd-78752bb4df42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:11.396012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:11.478728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ff33e-7c44-40f3-89ec-cef1f0d3b31a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:11.481377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1dc66-aee5-4610-98cd-78752bb4df42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:11.485137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:11.486098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:11.486384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:11.486709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] PATH_INFO: `/v3/projects/a78444d6606140808d457fc6016729e6` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:11.491054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a78444d6606140808d457fc6016729e6)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:11.493175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1dc66-aee5-4610-98cd-78752bb4df42 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:11.495968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:11.508233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 991/1962] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:11.563568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a78444d6606140808d457fc6016729e6', 'request_id': 'req-bc28dd31-f661-4350-a449-bca6ffa5d1e0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:11.572869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a78444d6606140808d457fc6016729e6', 'request_id': 'req-bc28dd31-f661-4350-a449-bca6ffa5d1e0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:11.604778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a78444d6606140808d457fc6016729e6', 'request_id': 'req-bc28dd31-f661-4350-a449-bca6ffa5d1e0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:11.620624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] The token cache is being invalidate because project a78444d6606140808d457fc6016729e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:11.621173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:11.622589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bc28dd31-f661-4350-a449-bca6ffa5d1e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:11.626035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 972/1963] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:11 2022] DELETE /identity/v3/projects/a78444d6606140808d457fc6016729e6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:11.630943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689def07-d1ad-4851-a581-51f68609f40c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:11.631731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689def07-d1ad-4851-a581-51f68609f40c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:11.632348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689def07-d1ad-4851-a581-51f68609f40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:11.660720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-689def07-d1ad-4851-a581-51f68609f40c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:11.662049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-689def07-d1ad-4851-a581-51f68609f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:11.734745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 992/1964] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:11.742525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f206-1814-4ca1-b7e8-9d1518791788 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:11.742822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f206-1814-4ca1-b7e8-9d1518791788 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:11.743103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f206-1814-4ca1-b7e8-9d1518791788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:11.762427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4af7f206-1814-4ca1-b7e8-9d1518791788 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:11.763987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4af7f206-1814-4ca1-b7e8-9d1518791788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:11.772617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 973/1965] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:11.776940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689def07-d1ad-4851-a581-51f68609f40c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:11.787245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689def07-d1ad-4851-a581-51f68609f40c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:11.792560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:11.793227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:11.793509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:11.793785 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:11.794631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:21:11.794939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:11.796595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:11.808195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 993/1966] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:21:11.812838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7f206-1814-4ca1-b7e8-9d1518791788 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:11.822601 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7f206-1814-4ca1-b7e8-9d1518791788 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:11.828951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2393-496a-478d-9768-b872bbe3946e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:11.829706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2393-496a-478d-9768-b872bbe3946e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:11.830378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2393-496a-478d-9768-b872bbe3946e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:11.830378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2393-496a-478d-9768-b872bbe3946e admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:11.830996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd2393-496a-478d-9768-b872bbe3946e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:11.833025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd2393-496a-478d-9768-b872bbe3946e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:11.854895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:11.863252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 974/1967] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:11 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:11.867904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:11.879448 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3bc8e-a53d-4c69-8af0-41ac02d8009f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:11.939180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f73c8-9013-4f14-ad2a-89be4fb70b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:11.939887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f73c8-9013-4f14-ad2a-89be4fb70b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:11.940168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f73c8-9013-4f14-ad2a-89be4fb70b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:11.940499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f73c8-9013-4f14-ad2a-89be4fb70b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:11.941207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f73c8-9013-4f14-ad2a-89be4fb70b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:11.943171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f73c8-9013-4f14-ad2a-89be4fb70b50 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:11.964150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 994/1968] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:11 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:11.968622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2393-496a-478d-9768-b872bbe3946e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:11.980015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2393-496a-478d-9768-b872bbe3946e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:11.985527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ba53-8098-4915-b304-1a50813de9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:11.986429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ba53-8098-4915-b304-1a50813de9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:11.986716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ba53-8098-4915-b304-1a50813de9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:11.986990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ba53-8098-4915-b304-1a50813de9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:11.987789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ba53-8098-4915-b304-1a50813de9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:11.989393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ba53-8098-4915-b304-1a50813de9b9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:12.001634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 975/1969] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:12.005572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f73c8-9013-4f14-ad2a-89be4fb70b50 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:12.018241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f73c8-9013-4f14-ad2a-89be4fb70b50 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:12.022897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b6073-4c23-4b7e-950a-399ae4480412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:12.023601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b6073-4c23-4b7e-950a-399ae4480412 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:12.023888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b6073-4c23-4b7e-950a-399ae4480412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:12.024165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b6073-4c23-4b7e-950a-399ae4480412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:12.024938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b6073-4c23-4b7e-950a-399ae4480412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:12.026559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b6073-4c23-4b7e-950a-399ae4480412 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:12.037997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 995/1970] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:12.042983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ba53-8098-4915-b304-1a50813de9b9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:12.054078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ba53-8098-4915-b304-1a50813de9b9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:12.059379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:12.060285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:12.060565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:12.060831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] PATH_INFO: `/v3/projects/a7def84912b34319bcdb7606ce55dd21/users/a02df01c3f054fcc8bad014ad8a65459/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:12.075795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7def84912b34319bcdb7606ce55dd21, user_id=a02df01c3f054fcc8bad014ad8a65459, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:12.078448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:12.095551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 976/1971] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:12 2022] PUT /identity/v3/projects/a7def84912b34319bcdb7606ce55dd21/users/a02df01c3f054fcc8bad014ad8a65459/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:12.101558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c97c6-cb4c-4646-8a76-f6ebbff68626 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:12.101970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c97c6-cb4c-4646-8a76-f6ebbff68626 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:12.102327 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c97c6-cb4c-4646-8a76-f6ebbff68626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:12.121730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6a3c97c6-cb4c-4646-8a76-f6ebbff68626 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:12.123538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6a3c97c6-cb4c-4646-8a76-f6ebbff68626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:12.127817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:12.183573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 996/1972] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:12.196731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:12.199613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:12.260751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb1b6f-96ed-4642-ac9a-55c64f113ff6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:12.265493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9198af-9498-4ba4-96b8-9c683692525f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:12.266148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9198af-9498-4ba4-96b8-9c683692525f service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:12.266463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9198af-9498-4ba4-96b8-9c683692525f service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:12.266739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9198af-9498-4ba4-96b8-9c683692525f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:12.277402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9198af-9498-4ba4-96b8-9c683692525f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:12.278967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9198af-9498-4ba4-96b8-9c683692525f service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:12.290105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 977/1973] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:13.122181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7cef9-53b0-4b13-b5fa-706d270abc80 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:13.122888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7cef9-53b0-4b13-b5fa-706d270abc80 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:13.123334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7cef9-53b0-4b13-b5fa-706d270abc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:13.151077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-35a7cef9-53b0-4b13-b5fa-706d270abc80 None None] MFA Rules not processed for user `a02df01c3f054fcc8bad014ad8a65459`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:13.152787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-35a7cef9-53b0-4b13-b5fa-706d270abc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:13.230254 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 997/1974] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:13.235030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9198af-9498-4ba4-96b8-9c683692525f service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:13.244987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9198af-9498-4ba4-96b8-9c683692525f service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:13.250250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081fe93-d2df-4309-a50a-b96dd95d4920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:13.250974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081fe93-d2df-4309-a50a-b96dd95d4920 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:13.251321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081fe93-d2df-4309-a50a-b96dd95d4920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:13.251635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081fe93-d2df-4309-a50a-b96dd95d4920 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:13.252327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081fe93-d2df-4309-a50a-b96dd95d4920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:13.254233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081fe93-d2df-4309-a50a-b96dd95d4920 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:13.284270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 978/1975] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:13 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:13.288410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7cef9-53b0-4b13-b5fa-706d270abc80 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:13.299113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7cef9-53b0-4b13-b5fa-706d270abc80 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:13.355305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6a0c7-e4b1-47c5-8b26-d26955f2560f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:13.355978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6a0c7-e4b1-47c5-8b26-d26955f2560f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:13.356293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6a0c7-e4b1-47c5-8b26-d26955f2560f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:13.356565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6a0c7-e4b1-47c5-8b26-d26955f2560f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:13.357385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6a0c7-e4b1-47c5-8b26-d26955f2560f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:13.359352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6a0c7-e4b1-47c5-8b26-d26955f2560f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:13.380736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 998/1976] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:13 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:13.386990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081fe93-d2df-4309-a50a-b96dd95d4920 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:13.403753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081fe93-d2df-4309-a50a-b96dd95d4920 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:13.413235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881e729-b4b9-4b9c-9202-9e231e228790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:13.414180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881e729-b4b9-4b9c-9202-9e231e228790 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:13.414564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881e729-b4b9-4b9c-9202-9e231e228790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:13.414906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881e729-b4b9-4b9c-9202-9e231e228790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:13.416069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881e729-b4b9-4b9c-9202-9e231e228790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:13.418544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881e729-b4b9-4b9c-9202-9e231e228790 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:13.435895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 979/1977] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:13.440342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6a0c7-e4b1-47c5-8b26-d26955f2560f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:13.450526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6a0c7-e4b1-47c5-8b26-d26955f2560f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:13.456006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b5109-8ded-4403-a66f-69f913f00cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:13.456885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b5109-8ded-4403-a66f-69f913f00cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:13.457163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b5109-8ded-4403-a66f-69f913f00cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:13.457426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b5109-8ded-4403-a66f-69f913f00cc9 admin admin] PATH_INFO: `/v3/projects/989ea056d48a4599b9e7582d9083be6a/users/9614419e50364a9381cf9a12aaa2eae0/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:13.485266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b5109-8ded-4403-a66f-69f913f00cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=989ea056d48a4599b9e7582d9083be6a, user_id=9614419e50364a9381cf9a12aaa2eae0, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:13.488015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b5109-8ded-4403-a66f-69f913f00cc9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:13.504754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 999/1978] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:13 2022] PUT /identity/v3/projects/989ea056d48a4599b9e7582d9083be6a/users/9614419e50364a9381cf9a12aaa2eae0/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:13.508957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881e729-b4b9-4b9c-9202-9e231e228790 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:13.519605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881e729-b4b9-4b9c-9202-9e231e228790 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:13.577886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccba20-7662-4b80-9750-e98e898c5f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:13.578593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccba20-7662-4b80-9750-e98e898c5f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:13.578874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccba20-7662-4b80-9750-e98e898c5f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:13.579147 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccba20-7662-4b80-9750-e98e898c5f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:13.579911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccba20-7662-4b80-9750-e98e898c5f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:13.581542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccba20-7662-4b80-9750-e98e898c5f05 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:13.593002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 980/1979] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:13.596908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b5109-8ded-4403-a66f-69f913f00cc9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:13.604305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:13.608863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b5109-8ded-4403-a66f-69f913f00cc9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:13.615197 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:13.615647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:13.615915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:13.616185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] PATH_INFO: `/v3/projects/989ea056d48a4599b9e7582d9083be6a/users/9614419e50364a9381cf9a12aaa2eae0/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:13.618409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=989ea056d48a4599b9e7582d9083be6a, user_id=9614419e50364a9381cf9a12aaa2eae0, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:13.621198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:13.633767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1000/1980] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:13 2022] PUT /identity/v3/projects/989ea056d48a4599b9e7582d9083be6a/users/9614419e50364a9381cf9a12aaa2eae0/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:14.549649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34756-10dc-4bae-8222-174823694054 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:14.550182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34756-10dc-4bae-8222-174823694054 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:14.550182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34756-10dc-4bae-8222-174823694054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:14.577028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2cb34756-10dc-4bae-8222-174823694054 None None] MFA Rules not processed for user `9614419e50364a9381cf9a12aaa2eae0`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:14.578295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2cb34756-10dc-4bae-8222-174823694054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:14.596617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:14.651482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 981/1981] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:14.663143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:14.665082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:14.724880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a7282-03d9-4e0b-a5f8-8063f48f3a13 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:14.729177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:14.729945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:14.730310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:14.730644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:14.795956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:14.797553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:14.808258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1001/1982] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:17.308273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb34756-10dc-4bae-8222-174823694054 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:17.309740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2cb34756-10dc-4bae-8222-174823694054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:17.323233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb34756-10dc-4bae-8222-174823694054 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:17.327339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169859f2-adf3-4d1f-b7a8-ddc2b154ee48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:17.327961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169859f2-adf3-4d1f-b7a8-ddc2b154ee48 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:17.328235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169859f2-adf3-4d1f-b7a8-ddc2b154ee48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:17.328543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169859f2-adf3-4d1f-b7a8-ddc2b154ee48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:17.337829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169859f2-adf3-4d1f-b7a8-ddc2b154ee48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:17.339279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169859f2-adf3-4d1f-b7a8-ddc2b154ee48 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:17.348730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 982/1983] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:24.052997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:24.054443 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:24.135411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3425f9-29b4-4870-bca5-6108d6d0d657 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:24.140658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:24.141673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:24.141954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:24.142225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] PATH_INFO: `/v3/users/bd3e7d31890b432ab2fb772501557c2e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:24.143515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd3e7d31890b432ab2fb772501557c2e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:24.145575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:24.194894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd3e7d31890b432ab2fb772501557c2e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.195562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd3e7d31890b432ab2fb772501557c2e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.196763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:24.204080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd3e7d31890b432ab2fb772501557c2e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.216188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd3e7d31890b432ab2fb772501557c2e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.224641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd3e7d31890b432ab2fb772501557c2e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.230707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1002/1984] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:24 2022] DELETE /identity/v3/users/bd3e7d31890b432ab2fb772501557c2e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:24.235910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0ef5-cb66-41f3-9448-1ebf6eb4d897 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:24.236416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0ef5-cb66-41f3-9448-1ebf6eb4d897 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:24.236581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0ef5-cb66-41f3-9448-1ebf6eb4d897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:24.255486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3ace0ef5-cb66-41f3-9448-1ebf6eb4d897 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:24.256701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3ace0ef5-cb66-41f3-9448-1ebf6eb4d897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:24.321261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 983/1985] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:24.330942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:24.332415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:24.392756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba91c1-fa67-4225-aa59-8a6c3e939f8a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:24.397181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927f2c6-220e-4837-b938-f22388726acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:24.397845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927f2c6-220e-4837-b938-f22388726acf service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:24.398133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927f2c6-220e-4837-b938-f22388726acf service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:24.398405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927f2c6-220e-4837-b938-f22388726acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:24.409933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927f2c6-220e-4837-b938-f22388726acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:24.411521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927f2c6-220e-4837-b938-f22388726acf service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:24.421640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1003/1986] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:24.762849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace0ef5-cb66-41f3-9448-1ebf6eb4d897 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:24.764704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3ace0ef5-cb66-41f3-9448-1ebf6eb4d897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:24.777090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace0ef5-cb66-41f3-9448-1ebf6eb4d897 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:24.791539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:24.792640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:24.792924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:24.793202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] PATH_INFO: `/v3/projects/5f83dba68b0d4d2c90a91e950ac9b183` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:24.794997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f83dba68b0d4d2c90a91e950ac9b183)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:24.797709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:24.860785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f83dba68b0d4d2c90a91e950ac9b183', 'request_id': 'req-26522926-5930-4bc8-b36f-f309294b1911'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.865448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f83dba68b0d4d2c90a91e950ac9b183', 'request_id': 'req-26522926-5930-4bc8-b36f-f309294b1911'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.874642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f83dba68b0d4d2c90a91e950ac9b183', 'request_id': 'req-26522926-5930-4bc8-b36f-f309294b1911'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.886315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] The token cache is being invalidate because project 5f83dba68b0d4d2c90a91e950ac9b183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:24.886604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.887277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:24.890743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 984/1987] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:24 2022] DELETE /identity/v3/projects/5f83dba68b0d4d2c90a91e950ac9b183 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:24.894633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927f2c6-220e-4837-b938-f22388726acf service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:24.906290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927f2c6-220e-4837-b938-f22388726acf service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:24.975845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:24.976734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:24.977011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:24.977314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] PATH_INFO: `/v3/users/db7ceb0af18549358f270780f74ec717` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:24.978496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7ceb0af18549358f270780f74ec717)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:24.980440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:25.045296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7ceb0af18549358f270780f74ec717'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.046033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7ceb0af18549358f270780f74ec717'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.053472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7ceb0af18549358f270780f74ec717'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.064051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:25.066816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7ceb0af18549358f270780f74ec717'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.075911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7ceb0af18549358f270780f74ec717'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.082632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1004/1988] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:24 2022] DELETE /identity/v3/users/db7ceb0af18549358f270780f74ec717 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:25.364743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:25.366269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:25.380333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26522926-5930-4bc8-b36f-f309294b1911 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:25.404474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:25.445098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:25.445926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:25.446208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:25.446488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] PATH_INFO: `/v3/projects/0ea26f9337134a25ae8cde6100ebd409` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:25.447693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea26f9337134a25ae8cde6100ebd409)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:25.449669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:25.500130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea26f9337134a25ae8cde6100ebd409', 'request_id': 'req-1a9673d8-3e47-4136-a91b-d6ad585de814'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.505756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea26f9337134a25ae8cde6100ebd409', 'request_id': 'req-1a9673d8-3e47-4136-a91b-d6ad585de814'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.514758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea26f9337134a25ae8cde6100ebd409', 'request_id': 'req-1a9673d8-3e47-4136-a91b-d6ad585de814'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.527079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] The token cache is being invalidate because project 0ea26f9337134a25ae8cde6100ebd409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:25.527432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.528188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.530050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 985/1989] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:25 2022] DELETE /identity/v3/projects/0ea26f9337134a25ae8cde6100ebd409 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:25.534200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:25.535616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:25.551429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa554c63-ce96-4893-8c6f-306036e8e45e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:25.613999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:25.614880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:25.615158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:25.615462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] PATH_INFO: `/v3/users/b388cb41f2e0408088bb8ee405619515` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:25.616626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b388cb41f2e0408088bb8ee405619515)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:25.618567 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:25.676117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b388cb41f2e0408088bb8ee405619515'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.676775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b388cb41f2e0408088bb8ee405619515'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.684208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b388cb41f2e0408088bb8ee405619515'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.696730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b388cb41f2e0408088bb8ee405619515'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.705925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b14fc7db-c8a9-44c4-bc47-67c6bb5b973a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b388cb41f2e0408088bb8ee405619515'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:25.712058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1005/1990] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:25 2022] DELETE /identity/v3/users/b388cb41f2e0408088bb8ee405619515 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:26.091128 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:26.092941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:26.112644 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9673d8-3e47-4136-a91b-d6ad585de814 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:26.174776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:26.175695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.175972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.176244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] PATH_INFO: `/v3/projects/81cdabe37b264181aeb1f18cf11a4551` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.177438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=81cdabe37b264181aeb1f18cf11a4551)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:26.179389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:26.226538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81cdabe37b264181aeb1f18cf11a4551', 'request_id': 'req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:26.231357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:26.232798 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81cdabe37b264181aeb1f18cf11a4551', 'request_id': 'req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:26.242545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81cdabe37b264181aeb1f18cf11a4551', 'request_id': 'req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:26.255556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] The token cache is being invalidate because project 81cdabe37b264181aeb1f18cf11a4551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:26.255842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:26.256484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a761106b-5fbd-4c7d-a65a-bc6a2aa37b7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:26.258046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 986/1991] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:26 2022] DELETE /identity/v3/projects/81cdabe37b264181aeb1f18cf11a4551 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:26.263353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af3ec8-3021-4c57-9fe4-d77439010110 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.263956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af3ec8-3021-4c57-9fe4-d77439010110 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.263956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af3ec8-3021-4c57-9fe4-d77439010110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.282857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-49af3ec8-3021-4c57-9fe4-d77439010110 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:26.284116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-49af3ec8-3021-4c57-9fe4-d77439010110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:26.352388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1006/1992] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:26.360900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dce2-39cf-4509-83b7-78c524c19516 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.361212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dce2-39cf-4509-83b7-78c524c19516 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.361485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dce2-39cf-4509-83b7-78c524c19516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.380482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-95b7dce2-39cf-4509-83b7-78c524c19516 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:26.381686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-95b7dce2-39cf-4509-83b7-78c524c19516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:26.389709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 987/1993] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:26.394046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af3ec8-3021-4c57-9fe4-d77439010110 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:26.403706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af3ec8-3021-4c57-9fe4-d77439010110 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:26.410165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:26.410834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.411110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.411412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.412139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:21:26.412435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:26.414048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:26.415765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:26.429111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1007/1994] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:21:26.433226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7dce2-39cf-4509-83b7-78c524c19516 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:26.442801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7dce2-39cf-4509-83b7-78c524c19516 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:26.447939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed7402-e9af-4dd1-8403-88f3304d0e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:26.448684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed7402-e9af-4dd1-8403-88f3304d0e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.449022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed7402-e9af-4dd1-8403-88f3304d0e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.449302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed7402-e9af-4dd1-8403-88f3304d0e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.450081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed7402-e9af-4dd1-8403-88f3304d0e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:26.452114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed7402-e9af-4dd1-8403-88f3304d0e81 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:26.479843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 988/1995] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:26 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:26.484604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:26.495307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37f324-35f5-4953-91f1-acea0da80a5c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:26.555429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6145d2-a6c5-4366-bcb2-4c50c34c9316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:26.556106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6145d2-a6c5-4366-bcb2-4c50c34c9316 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.556390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6145d2-a6c5-4366-bcb2-4c50c34c9316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.556665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6145d2-a6c5-4366-bcb2-4c50c34c9316 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.557368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6145d2-a6c5-4366-bcb2-4c50c34c9316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:26.559345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6145d2-a6c5-4366-bcb2-4c50c34c9316 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:26.581162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1008/1996] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:26 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:26.585372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed7402-e9af-4dd1-8403-88f3304d0e81 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:26.596011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed7402-e9af-4dd1-8403-88f3304d0e81 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:26.601814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfa956-b014-415b-8b0c-071dd3dbcafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:26.602942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfa956-b014-415b-8b0c-071dd3dbcafa admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.603469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfa956-b014-415b-8b0c-071dd3dbcafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.603979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfa956-b014-415b-8b0c-071dd3dbcafa admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.605295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bfa956-b014-415b-8b0c-071dd3dbcafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:26.608581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bfa956-b014-415b-8b0c-071dd3dbcafa admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:26.620182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 989/1997] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:26.625140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6145d2-a6c5-4366-bcb2-4c50c34c9316 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:26.635252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6145d2-a6c5-4366-bcb2-4c50c34c9316 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:26.640977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7934e4-d284-4b9a-9e6f-f4a4158b602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:26.641646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7934e4-d284-4b9a-9e6f-f4a4158b602f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.641922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7934e4-d284-4b9a-9e6f-f4a4158b602f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.642192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7934e4-d284-4b9a-9e6f-f4a4158b602f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.642930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7934e4-d284-4b9a-9e6f-f4a4158b602f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:26.644566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7934e4-d284-4b9a-9e6f-f4a4158b602f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:26.657915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1009/1998] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:26.663634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfa956-b014-415b-8b0c-071dd3dbcafa admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:26.674102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfa956-b014-415b-8b0c-071dd3dbcafa admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:26.678894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:26.679825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.680112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.680389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] PATH_INFO: `/v3/projects/bea8f88d81c04623b66edaf1cebc2ee3/users/44b66024bd78487082154441343a2161/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.696732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea8f88d81c04623b66edaf1cebc2ee3, user_id=44b66024bd78487082154441343a2161, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:26.699435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:26.715398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 990/1999] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:26 2022] PUT /identity/v3/projects/bea8f88d81c04623b66edaf1cebc2ee3/users/44b66024bd78487082154441343a2161/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:26.721377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52cbc2-e1c4-4e56-9803-6717608b0d3c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.721667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52cbc2-e1c4-4e56-9803-6717608b0d3c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.721942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52cbc2-e1c4-4e56-9803-6717608b0d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.739923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc52cbc2-e1c4-4e56-9803-6717608b0d3c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:26.741116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc52cbc2-e1c4-4e56-9803-6717608b0d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:26.802179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1010/2000] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:26.812772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:26.815957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:26.880750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd36c3-e200-47d3-9429-37ffec70cb8b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:26.885338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:26.885986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:26.886265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:26.886538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:26.896319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:26.897760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:26.907857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 991/2001] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:27.723451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47a27-c5d7-4aaf-b356-2c6697823db6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:27.723977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47a27-c5d7-4aaf-b356-2c6697823db6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:27.724064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47a27-c5d7-4aaf-b356-2c6697823db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:27.750796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-82d47a27-c5d7-4aaf-b356-2c6697823db6 None None] MFA Rules not processed for user `44b66024bd78487082154441343a2161`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:27.752046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-82d47a27-c5d7-4aaf-b356-2c6697823db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:27.817108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1011/2002] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:27.828113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:27.829579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:27.887757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c95421-31ea-4b2f-ab4c-73a6db73722e service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:27.892551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:27.893199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:27.893759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:27.894043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:27.902766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:27.904241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:27.913354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 992/2003] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:35.644076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5978-5845-4127-a69e-e6330c82e070 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:35.644727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5978-5845-4127-a69e-e6330c82e070 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:35.644727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5978-5845-4127-a69e-e6330c82e070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:35.653606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:35.672474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cafb5978-5845-4127-a69e-e6330c82e070 None None] MFA Rules not processed for user `44b66024bd78487082154441343a2161`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:35.674074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cafb5978-5845-4127-a69e-e6330c82e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:35.682284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1012/2004] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:21:37.909382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:45.670274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:53.783344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:53.786385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:53.803330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5460e52-03b7-41ef-9d8d-0f38e10282d8 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:53.808267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:53.809583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:53.809583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:53.809801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] PATH_INFO: `/v3/users/c62a18ec68d5494e912f101ee9b1ebad` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:53.810922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62a18ec68d5494e912f101ee9b1ebad)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:53.812926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:53.861318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62a18ec68d5494e912f101ee9b1ebad'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:53.870026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62a18ec68d5494e912f101ee9b1ebad'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:53.878009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62a18ec68d5494e912f101ee9b1ebad'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:53.878684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62a18ec68d5494e912f101ee9b1ebad'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:53.891413 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62a18ec68d5494e912f101ee9b1ebad'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:53.897578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 993/2005] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:53 2022] DELETE /identity/v3/users/c62a18ec68d5494e912f101ee9b1ebad => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:53.903407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b855-e345-4bd6-b242-e18bf01f0270 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:53.903958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b855-e345-4bd6-b242-e18bf01f0270 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:53.903958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b855-e345-4bd6-b242-e18bf01f0270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:53.921950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9a47b855-e345-4bd6-b242-e18bf01f0270 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:53.923131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9a47b855-e345-4bd6-b242-e18bf01f0270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:53.984793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1013/2006] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:53.994292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:53.995752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:54.055136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6572f2d-c6df-4add-aac6-526270d855ef admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:54.059110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d47c3-a861-4dbb-816f-5246ad4e9d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:54.059774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d47c3-a861-4dbb-816f-5246ad4e9d4e service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:54.060086 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d47c3-a861-4dbb-816f-5246ad4e9d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:54.060338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d47c3-a861-4dbb-816f-5246ad4e9d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:54.069986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d47c3-a861-4dbb-816f-5246ad4e9d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:54.071484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d47c3-a861-4dbb-816f-5246ad4e9d4e service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:54.081521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 994/2007] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:54.152383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdab1d-45c0-46a8-8ba9-f9d1ebf2e052 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:54.153041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdab1d-45c0-46a8-8ba9-f9d1ebf2e052 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:54.153400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdab1d-45c0-46a8-8ba9-f9d1ebf2e052 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:54.157069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1014/2008] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:21:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:21:54.167480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81d0ce-ac24-4aa5-a06e-efbd2683a702 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:54.167755 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81d0ce-ac24-4aa5-a06e-efbd2683a702 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:54.168035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81d0ce-ac24-4aa5-a06e-efbd2683a702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:54.190713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cf81d0ce-ac24-4aa5-a06e-efbd2683a702 None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:54.191948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cf81d0ce-ac24-4aa5-a06e-efbd2683a702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:54.276741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 995/2009] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:21:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:54.290822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdab1d-45c0-46a8-8ba9-f9d1ebf2e052 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:54.293136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-acbdab1d-45c0-46a8-8ba9-f9d1ebf2e052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:54.312259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdab1d-45c0-46a8-8ba9-f9d1ebf2e052 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:54.317518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:54.318627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:54.319100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:54.319598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:54.331152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:54.333065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:54.344701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1015/2010] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:54.876199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81d0ce-ac24-4aa5-a06e-efbd2683a702 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:54.877719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cf81d0ce-ac24-4aa5-a06e-efbd2683a702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:54.894073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81d0ce-ac24-4aa5-a06e-efbd2683a702 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:54.900051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:54.901449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:54.901915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:54.902374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] PATH_INFO: `/v3/projects/bdbc517ae372470e87e5e73c1f326798` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:54.905260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdbc517ae372470e87e5e73c1f326798)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:54.909800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:54.910883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:54.912355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:54.927033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:54.930224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764526-d3b4-408c-8831-1fcac004578b service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:54.938499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:54.939521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:54.940042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:54.940537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] PATH_INFO: `/v3/users/a02df01c3f054fcc8bad014ad8a65459` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:54.941806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02df01c3f054fcc8bad014ad8a65459)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:54.944037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:54.987971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdbc517ae372470e87e5e73c1f326798', 'request_id': 'req-56024669-5b20-4786-9383-6f6acd9ba0ec'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.000430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdbc517ae372470e87e5e73c1f326798', 'request_id': 'req-56024669-5b20-4786-9383-6f6acd9ba0ec'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.011863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdbc517ae372470e87e5e73c1f326798', 'request_id': 'req-56024669-5b20-4786-9383-6f6acd9ba0ec'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.015520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02df01c3f054fcc8bad014ad8a65459'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.016472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02df01c3f054fcc8bad014ad8a65459'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.027072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02df01c3f054fcc8bad014ad8a65459'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.029536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] The token cache is being invalidate because project bdbc517ae372470e87e5e73c1f326798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:55.029996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.031425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56024669-5b20-4786-9383-6f6acd9ba0ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.034324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 996/2011] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:54 2022] DELETE /identity/v3/projects/bdbc517ae372470e87e5e73c1f326798 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:55.041765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396df9b-815c-4aec-a06d-28bdda81744b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.042059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396df9b-815c-4aec-a06d-28bdda81744b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.042335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396df9b-815c-4aec-a06d-28bdda81744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.045299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02df01c3f054fcc8bad014ad8a65459'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.054932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-08d3ec95-e3d7-429e-bd9b-86db5b7684a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02df01c3f054fcc8bad014ad8a65459'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.064462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d396df9b-815c-4aec-a06d-28bdda81744b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:55.065969 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d396df9b-815c-4aec-a06d-28bdda81744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:55.067267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1016/2012] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:54 2022] DELETE /identity/v3/users/a02df01c3f054fcc8bad014ad8a65459 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:55.164703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 997/2013] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:55.173012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538188be-6c60-4520-8691-14018032b7b0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.173389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538188be-6c60-4520-8691-14018032b7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.173858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538188be-6c60-4520-8691-14018032b7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.196579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-538188be-6c60-4520-8691-14018032b7b0 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:55.197954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-538188be-6c60-4520-8691-14018032b7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:55.218854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1017/2014] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:55.220232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396df9b-815c-4aec-a06d-28bdda81744b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.230487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396df9b-815c-4aec-a06d-28bdda81744b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.237397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:55.237990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.238774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.239029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.241545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:21:55.241870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:55.248903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:55.266004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 998/2015] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:21:55.267227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538188be-6c60-4520-8691-14018032b7b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.278910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538188be-6c60-4520-8691-14018032b7b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.281599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.283932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:55.288222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c393a1-4d9a-4e44-9536-8a98790609e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:55.289886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c393a1-4d9a-4e44-9536-8a98790609e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.290568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c393a1-4d9a-4e44-9536-8a98790609e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.291045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c393a1-4d9a-4e44-9536-8a98790609e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.292211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c393a1-4d9a-4e44-9536-8a98790609e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:55.294522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c393a1-4d9a-4e44-9536-8a98790609e6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:55.309769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51967ee9-18ed-463a-bdf6-819e41555fe3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.321225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:55.322203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.322528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.322811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] PATH_INFO: `/v3/projects/a7def84912b34319bcdb7606ce55dd21` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.324058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7def84912b34319bcdb7606ce55dd21)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:55.332712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:55.340502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1018/2016] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:55 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:55.347916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c393a1-4d9a-4e44-9536-8a98790609e6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.388943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:55.409020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7def84912b34319bcdb7606ce55dd21', 'request_id': 'req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.414518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7def84912b34319bcdb7606ce55dd21', 'request_id': 'req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.423526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7def84912b34319bcdb7606ce55dd21', 'request_id': 'req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.423710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c393a1-4d9a-4e44-9536-8a98790609e6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.436462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] The token cache is being invalidate because project a7def84912b34319bcdb7606ce55dd21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:55.436751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.437438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.439289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 999/2017] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:55 2022] DELETE /identity/v3/projects/a7def84912b34319bcdb7606ce55dd21 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:55.442955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.444320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:55.450787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:55.480421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:55.481090 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.481369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.481642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.482423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:55.484781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:55.512755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1019/2018] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:55 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:55.517437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.520167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:55.543104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089d7c5-8274-463b-a56c-f63f6dae4807 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.548487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afc6f5-59db-4fd9-a1ea-6a30e77e3aa5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.552549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b038321-4ebb-44e5-bce4-1a6974d40a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:55.552549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b038321-4ebb-44e5-bce4-1a6974d40a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.553971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b038321-4ebb-44e5-bce4-1a6974d40a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.554438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b038321-4ebb-44e5-bce4-1a6974d40a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.555907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b038321-4ebb-44e5-bce4-1a6974d40a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:55.556933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:55.558478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b038321-4ebb-44e5-bce4-1a6974d40a1c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:55.558691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.559244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.559793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] PATH_INFO: `/v3/users/9614419e50364a9381cf9a12aaa2eae0` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.561857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9614419e50364a9381cf9a12aaa2eae0)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:55.565517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:55.579632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1020/2019] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:21:55.584393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b038321-4ebb-44e5-bce4-1a6974d40a1c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.598704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b038321-4ebb-44e5-bce4-1a6974d40a1c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.604280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d328-f870-4dfc-a559-50e9cd0ea38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:55.605117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d328-f870-4dfc-a559-50e9cd0ea38c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.605474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d328-f870-4dfc-a559-50e9cd0ea38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.605814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d328-f870-4dfc-a559-50e9cd0ea38c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.606883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779d328-f870-4dfc-a559-50e9cd0ea38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:55.608751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779d328-f870-4dfc-a559-50e9cd0ea38c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:55.621302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1021/2020] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:55.625797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d328-f870-4dfc-a559-50e9cd0ea38c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.635835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d328-f870-4dfc-a559-50e9cd0ea38c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.636290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9614419e50364a9381cf9a12aaa2eae0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.640694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:55.641596 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.641874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.642146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] PATH_INFO: `/v3/projects/b574edec2fec48a98bc1c31cfc58afaa/users/45b8a18312134372a867a578f1cf55f3/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.646995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9614419e50364a9381cf9a12aaa2eae0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.657541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9614419e50364a9381cf9a12aaa2eae0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.658264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9614419e50364a9381cf9a12aaa2eae0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.659585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b574edec2fec48a98bc1c31cfc58afaa, user_id=45b8a18312134372a867a578f1cf55f3, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:55.662644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:55.672977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0adca1-d925-4dc6-8b3a-007e6b4a3657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9614419e50364a9381cf9a12aaa2eae0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:55.679670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1000/2021] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:55 2022] DELETE /identity/v3/users/9614419e50364a9381cf9a12aaa2eae0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:55.683742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1022/2022] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:55 2022] PUT /identity/v3/projects/b574edec2fec48a98bc1c31cfc58afaa/users/45b8a18312134372a867a578f1cf55f3/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:55.694466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3ea15-c996-4272-932c-4fae1e6e007f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:55.694818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3ea15-c996-4272-932c-4fae1e6e007f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:55.695101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3ea15-c996-4272-932c-4fae1e6e007f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:55.726163 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-dbb3ea15-c996-4272-932c-4fae1e6e007f None None] MFA Rules not processed for user `45b8a18312134372a867a578f1cf55f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:55.727767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dbb3ea15-c996-4272-932c-4fae1e6e007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:55.819121 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1001/2023] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:55.834202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.840846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:55.951058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cb773-07bc-489d-b53c-f8fae422e425 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:55.959469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3ea15-c996-4272-932c-4fae1e6e007f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:55.962237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dbb3ea15-c996-4272-932c-4fae1e6e007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:55.989695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:56.042232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:56.069932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fce89b-f7e8-41b3-9c5f-82264ef9cfe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.070883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fce89b-f7e8-41b3-9c5f-82264ef9cfe2 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.071580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fce89b-f7e8-41b3-9c5f-82264ef9cfe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.071580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fce89b-f7e8-41b3-9c5f-82264ef9cfe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.085253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fce89b-f7e8-41b3-9c5f-82264ef9cfe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.086732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fce89b-f7e8-41b3-9c5f-82264ef9cfe2 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.100964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3ea15-c996-4272-932c-4fae1e6e007f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.101815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1023/2024] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:56.106102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.107788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.107788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.107788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] PATH_INFO: `/v3/projects/989ea056d48a4599b9e7582d9083be6a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.108952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=989ea056d48a4599b9e7582d9083be6a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.111012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.218278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '989ea056d48a4599b9e7582d9083be6a', 'request_id': 'req-e3b67150-9d1c-4d75-9832-297eae0be4f9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.225283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '989ea056d48a4599b9e7582d9083be6a', 'request_id': 'req-e3b67150-9d1c-4d75-9832-297eae0be4f9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.241856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '989ea056d48a4599b9e7582d9083be6a', 'request_id': 'req-e3b67150-9d1c-4d75-9832-297eae0be4f9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.263363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] The token cache is being invalidate because project 989ea056d48a4599b9e7582d9083be6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:56.264032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.265129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.273988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c4477-8add-4b34-81ff-82a07200326b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.275009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c4477-8add-4b34-81ff-82a07200326b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.275483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c4477-8add-4b34-81ff-82a07200326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.278212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1002/2025] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:55 2022] DELETE /identity/v3/projects/989ea056d48a4599b9e7582d9083be6a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:56.301314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-313c4477-8add-4b34-81ff-82a07200326b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:56.302795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-313c4477-8add-4b34-81ff-82a07200326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:56.354751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:56.356665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:56.383018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1024/2026] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:56.393251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadea05-d972-43f0-8309-2387f222d24f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.393636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadea05-d972-43f0-8309-2387f222d24f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.394050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadea05-d972-43f0-8309-2387f222d24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.455643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7aadea05-d972-43f0-8309-2387f222d24f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:56.456968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7aadea05-d972-43f0-8309-2387f222d24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:56.462881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b67150-9d1c-4d75-9832-297eae0be4f9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.482111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1025/2027] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:56.483701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.486241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadea05-d972-43f0-8309-2387f222d24f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:56.486589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.486898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.487183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] PATH_INFO: `/v3/users/c77ad5458095401b88b7bf101eadacc7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.493998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77ad5458095401b88b7bf101eadacc7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.500670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.507653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadea05-d972-43f0-8309-2387f222d24f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.522231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.523141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.523501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.523828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.524575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:21:56.524971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.533777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.555472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1026/2028] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:21:56.560048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:56.571360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2358735-b83a-4abf-88e8-f83c522a88db admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.577747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82431dae-8e54-48f2-94e0-c548d596c129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.578647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82431dae-8e54-48f2-94e0-c548d596c129 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.579029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82431dae-8e54-48f2-94e0-c548d596c129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.579473 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82431dae-8e54-48f2-94e0-c548d596c129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.580628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82431dae-8e54-48f2-94e0-c548d596c129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.582805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82431dae-8e54-48f2-94e0-c548d596c129 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.613174 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:56.616152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77ad5458095401b88b7bf101eadacc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.638168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77ad5458095401b88b7bf101eadacc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.645531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1027/2029] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:56 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:56.651649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82431dae-8e54-48f2-94e0-c548d596c129 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:56.651813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77ad5458095401b88b7bf101eadacc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.652818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77ad5458095401b88b7bf101eadacc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.674360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ec2c2318-48e1-41ef-ab3c-0457f1efd42b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77ad5458095401b88b7bf101eadacc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:56.692853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1003/2030] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:56 2022] DELETE /identity/v3/users/c77ad5458095401b88b7bf101eadacc7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:56.697671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca3295-839b-4b38-94c9-c482b977775e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.698105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca3295-839b-4b38-94c9-c482b977775e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.698556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca3295-839b-4b38-94c9-c482b977775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.720678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7aca3295-839b-4b38-94c9-c482b977775e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:56.722316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7aca3295-839b-4b38-94c9-c482b977775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:56.745713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82431dae-8e54-48f2-94e0-c548d596c129 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.802786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1004/2031] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:56.803511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:56.814432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca3295-839b-4b38-94c9-c482b977775e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:56.816379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7aca3295-839b-4b38-94c9-c482b977775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:56.825572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.826470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.826669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.826939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.827649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.829631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.859677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1028/2032] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:56 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:56.860863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:56.862939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:56.876986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f99e-0fd9-44df-a797-469c6416acf7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.883646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9539f2-8942-40ed-aee6-cc626c0ee04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.884625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9539f2-8942-40ed-aee6-cc626c0ee04f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.884875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9539f2-8942-40ed-aee6-cc626c0ee04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.885218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9539f2-8942-40ed-aee6-cc626c0ee04f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.885865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca3295-839b-4b38-94c9-c482b977775e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.886198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9539f2-8942-40ed-aee6-cc626c0ee04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.887910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9539f2-8942-40ed-aee6-cc626c0ee04f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.894899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02ed2f-181f-4597-87e1-6c2ed3b673bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.895644 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02ed2f-181f-4597-87e1-6c2ed3b673bc service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.895951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02ed2f-181f-4597-87e1-6c2ed3b673bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.896225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02ed2f-181f-4597-87e1-6c2ed3b673bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.899832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1029/2033] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:56.906345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9539f2-8942-40ed-aee6-cc626c0ee04f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:56.908347 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02ed2f-181f-4597-87e1-6c2ed3b673bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.909824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02ed2f-181f-4597-87e1-6c2ed3b673bc service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.917446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9539f2-8942-40ed-aee6-cc626c0ee04f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.920819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1005/2034] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:21:56.925872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d18de-abb0-48d5-addb-c3daec600951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.926750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d18de-abb0-48d5-addb-c3daec600951 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.927028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d18de-abb0-48d5-addb-c3daec600951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.927335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d18de-abb0-48d5-addb-c3daec600951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.928077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d18de-abb0-48d5-addb-c3daec600951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.930406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d18de-abb0-48d5-addb-c3daec600951 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:56.942241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1030/2035] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:21:56.948105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02ed2f-181f-4597-87e1-6c2ed3b673bc service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:56.959387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02ed2f-181f-4597-87e1-6c2ed3b673bc service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:56.964424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d5e78-c512-419b-bd17-d38785e370d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:56.965674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d5e78-c512-419b-bd17-d38785e370d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:56.965674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d5e78-c512-419b-bd17-d38785e370d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:56.965895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d5e78-c512-419b-bd17-d38785e370d0 admin admin] PATH_INFO: `/v3/projects/634bcb327f294ad08e43faf2c3d21d64/users/9a6df2f12caa4c99806ef20a6b91ffc7/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:56.988486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d5e78-c512-419b-bd17-d38785e370d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=634bcb327f294ad08e43faf2c3d21d64, user_id=9a6df2f12caa4c99806ef20a6b91ffc7, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:56.991534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d5e78-c512-419b-bd17-d38785e370d0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:57.013832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1006/2036] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:56 2022] PUT /identity/v3/projects/634bcb327f294ad08e43faf2c3d21d64/users/9a6df2f12caa4c99806ef20a6b91ffc7/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:57.026046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9ae7-20b5-44c0-803d-13d10302e918 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.026701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9ae7-20b5-44c0-803d-13d10302e918 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.026701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9ae7-20b5-44c0-803d-13d10302e918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.067164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6d8a9ae7-20b5-44c0-803d-13d10302e918 None None] MFA Rules not processed for user `9a6df2f12caa4c99806ef20a6b91ffc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:57.069684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6d8a9ae7-20b5-44c0-803d-13d10302e918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:57.186178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1031/2037] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:57.191300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d5e78-c512-419b-bd17-d38785e370d0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:57.262360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:57.302777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d5e78-c512-419b-bd17-d38785e370d0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:57.308272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ddd10-9ea9-40ae-b4e3-d79d186fe05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:57.311616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ddd10-9ea9-40ae-b4e3-d79d186fe05d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.312020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ddd10-9ea9-40ae-b4e3-d79d186fe05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.312449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ddd10-9ea9-40ae-b4e3-d79d186fe05d admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.313281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ddd10-9ea9-40ae-b4e3-d79d186fe05d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:57.315562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ddd10-9ea9-40ae-b4e3-d79d186fe05d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:57.341006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a9ae7-20b5-44c0-803d-13d10302e918 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:57.346785 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6d8a9ae7-20b5-44c0-803d-13d10302e918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:57.359695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1007/2038] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:57 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:57.363966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ddd10-9ea9-40ae-b4e3-d79d186fe05d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:57.381254 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a9ae7-20b5-44c0-803d-13d10302e918 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:57.463548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:57.464601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.464996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.465279 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] PATH_INFO: `/v3/projects/56e14ca337a749d7a90d794bcc45b01e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.466882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e14ca337a749d7a90d794bcc45b01e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:57.467961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ddd10-9ea9-40ae-b4e3-d79d186fe05d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:57.468976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:57.473196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb053a9-a764-4885-9c9a-b6204055b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:57.474026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb053a9-a764-4885-9c9a-b6204055b277 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.474402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb053a9-a764-4885-9c9a-b6204055b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.474770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb053a9-a764-4885-9c9a-b6204055b277 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.475677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb053a9-a764-4885-9c9a-b6204055b277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:57.477764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb053a9-a764-4885-9c9a-b6204055b277 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:57.505878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1008/2039] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:57 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:57.511170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb053a9-a764-4885-9c9a-b6204055b277 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:57.524537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb053a9-a764-4885-9c9a-b6204055b277 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:57.529802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5b62e-fd0d-463e-a603-9c3144dc6ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:57.530567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b62e-fd0d-463e-a603-9c3144dc6ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.530849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b62e-fd0d-463e-a603-9c3144dc6ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.531114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b62e-fd0d-463e-a603-9c3144dc6ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.532033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5b62e-fd0d-463e-a603-9c3144dc6ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:57.533524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5b62e-fd0d-463e-a603-9c3144dc6ddb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:57.545779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1009/2040] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:21:57.547191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:57.549863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5b62e-fd0d-463e-a603-9c3144dc6ddb admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:57.552855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e14ca337a749d7a90d794bcc45b01e', 'request_id': 'req-33933d60-75a8-46a2-b8ef-7da12d097d64'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.564975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5b62e-fd0d-463e-a603-9c3144dc6ddb admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:57.566620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e14ca337a749d7a90d794bcc45b01e', 'request_id': 'req-33933d60-75a8-46a2-b8ef-7da12d097d64'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.579303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e14ca337a749d7a90d794bcc45b01e', 'request_id': 'req-33933d60-75a8-46a2-b8ef-7da12d097d64'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.609275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] The token cache is being invalidate because project 56e14ca337a749d7a90d794bcc45b01e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:57.611754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.612409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.615922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1032/2041] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:57 2022] DELETE /identity/v3/projects/56e14ca337a749d7a90d794bcc45b01e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:57.619796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:57.621280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:57.676530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:57.677356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.677834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.678295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.679380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:57.681669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:57.701994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1010/2042] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:57.708467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:57.708763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33933d60-75a8-46a2-b8ef-7da12d097d64 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:57.710464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:57.716231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:57.717823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.718300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.718766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] PATH_INFO: `/v3/users/e1cd69c0d51f449eaa4168d30bc77084` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.721002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1cd69c0d51f449eaa4168d30bc77084)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:57.724381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff569edd-3600-4f62-ad3a-602d13e4c5dc admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:57.724661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:57.730852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf294b-e136-4c24-961a-108a055991b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:57.731933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf294b-e136-4c24-961a-108a055991b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.732386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf294b-e136-4c24-961a-108a055991b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.732895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf294b-e136-4c24-961a-108a055991b7 admin admin] PATH_INFO: `/v3/projects/c60bde0b6ac94b17bcc2ce5da133bbd7/users/e1500a9020714f5994d9adabed528f28/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.753995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf294b-e136-4c24-961a-108a055991b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60bde0b6ac94b17bcc2ce5da133bbd7, user_id=e1500a9020714f5994d9adabed528f28, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:57.756749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf294b-e136-4c24-961a-108a055991b7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:57.775778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1011/2043] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:57 2022] PUT /identity/v3/projects/c60bde0b6ac94b17bcc2ce5da133bbd7/users/e1500a9020714f5994d9adabed528f28/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:57.786966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a98b-ad79-445c-a002-929a556c2bf2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.787359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a98b-ad79-445c-a002-929a556c2bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.787740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a98b-ad79-445c-a002-929a556c2bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.787850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1cd69c0d51f449eaa4168d30bc77084'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.788471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1cd69c0d51f449eaa4168d30bc77084'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.797654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1cd69c0d51f449eaa4168d30bc77084'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.816774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1cd69c0d51f449eaa4168d30bc77084'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.820338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d746a98b-ad79-445c-a002-929a556c2bf2 None None] MFA Rules not processed for user `e1500a9020714f5994d9adabed528f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:57.821731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d746a98b-ad79-445c-a002-929a556c2bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:57.827798 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:57.830241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1cd69c0d51f449eaa4168d30bc77084'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:57.838312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1033/2044] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:21:57 2022] DELETE /identity/v3/users/e1cd69c0d51f449eaa4168d30bc77084 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:57.904178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1012/2045] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:57.909374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:57.910846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:57.972044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:57.987618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3976c8-d522-4b56-aa29-8e30ac86d3c3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:57.994430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:57.995165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:57.995655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:57.995991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:57.996713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:57.999325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.045439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1034/2046] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:57 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:58.050101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746a98b-ad79-445c-a002-929a556c2bf2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.164579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746a98b-ad79-445c-a002-929a556c2bf2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.166292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.170281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:58.244200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c706-a364-4037-8f22-bebffbd6912c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.246332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f71bd5-ec51-4345-b3c1-42adc59afca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.247047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f71bd5-ec51-4345-b3c1-42adc59afca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.247368 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f71bd5-ec51-4345-b3c1-42adc59afca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.248056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f71bd5-ec51-4345-b3c1-42adc59afca0 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.248903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f71bd5-ec51-4345-b3c1-42adc59afca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.249435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.250302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.250583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.250858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] PATH_INFO: `/v3/projects/cf39db6c918842a69cf76c0b52a59028` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.251367 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f71bd5-ec51-4345-b3c1-42adc59afca0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.252107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf39db6c918842a69cf76c0b52a59028)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.254022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.276878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1013/2047] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:58 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:58.277881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f71bd5-ec51-4345-b3c1-42adc59afca0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.291564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f71bd5-ec51-4345-b3c1-42adc59afca0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.296500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b50d-b656-4240-a38b-2c104a39df9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.297303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b50d-b656-4240-a38b-2c104a39df9d admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.297585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b50d-b656-4240-a38b-2c104a39df9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.297858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b50d-b656-4240-a38b-2c104a39df9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.298793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689b50d-b656-4240-a38b-2c104a39df9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.300571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689b50d-b656-4240-a38b-2c104a39df9d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.309458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf39db6c918842a69cf76c0b52a59028', 'request_id': 'req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:58.311524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1014/2048] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:21:58.315620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b50d-b656-4240-a38b-2c104a39df9d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.318379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf39db6c918842a69cf76c0b52a59028', 'request_id': 'req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:58.325653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b50d-b656-4240-a38b-2c104a39df9d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.326852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf39db6c918842a69cf76c0b52a59028', 'request_id': 'req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:58.338681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] The token cache is being invalidate because project cf39db6c918842a69cf76c0b52a59028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:21:58.338968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:58.339756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b56ec7b-b73b-483b-b639-69b22d3ee0bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:21:58.341769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1035/2049] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:21:58 2022] DELETE /identity/v3/projects/cf39db6c918842a69cf76c0b52a59028 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:58.346195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080eea5-bab2-4b8b-9f89-2a91630481c2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.346488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080eea5-bab2-4b8b-9f89-2a91630481c2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.346770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080eea5-bab2-4b8b-9f89-2a91630481c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.357846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:58.367949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7080eea5-bab2-4b8b-9f89-2a91630481c2 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:58.369206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7080eea5-bab2-4b8b-9f89-2a91630481c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:58.391988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fede3a-4951-46f1-b85f-e8eedb45f691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.393619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fede3a-4951-46f1-b85f-e8eedb45f691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.393911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fede3a-4951-46f1-b85f-e8eedb45f691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.394192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fede3a-4951-46f1-b85f-e8eedb45f691 admin admin] PATH_INFO: `/v3/projects/d5d44767e53448818c9f747c83df0024/users/04753e4485014154b21f75e3dd948d60/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.413229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fede3a-4951-46f1-b85f-e8eedb45f691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d44767e53448818c9f747c83df0024, user_id=04753e4485014154b21f75e3dd948d60, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.419146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fede3a-4951-46f1-b85f-e8eedb45f691 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.437406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1036/2050] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:58.440428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1015/2051] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:58 2022] PUT /identity/v3/projects/d5d44767e53448818c9f747c83df0024/users/04753e4485014154b21f75e3dd948d60/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:58.444420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080eea5-bab2-4b8b-9f89-2a91630481c2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.448186 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da356ee7-4681-477d-a7e9-66a8ee525779 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.448406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7080eea5-bab2-4b8b-9f89-2a91630481c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:58.448909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da356ee7-4681-477d-a7e9-66a8ee525779 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.449126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da356ee7-4681-477d-a7e9-66a8ee525779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.462020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080eea5-bab2-4b8b-9f89-2a91630481c2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.468937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:58.470333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-da356ee7-4681-477d-a7e9-66a8ee525779 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:58.471739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-da356ee7-4681-477d-a7e9-66a8ee525779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:58.527903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2225d-ec42-4f52-b25d-7bbf8d1c6fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.528703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2225d-ec42-4f52-b25d-7bbf8d1c6fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.529128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2225d-ec42-4f52-b25d-7bbf8d1c6fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.529523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2225d-ec42-4f52-b25d-7bbf8d1c6fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.530425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2225d-ec42-4f52-b25d-7bbf8d1c6fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.532134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2225d-ec42-4f52-b25d-7bbf8d1c6fd0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.542541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1037/2052] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:21:58.546486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2225d-ec42-4f52-b25d-7bbf8d1c6fd0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.549257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1016/2053] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:58.553610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da356ee7-4681-477d-a7e9-66a8ee525779 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.556939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2225d-ec42-4f52-b25d-7bbf8d1c6fd0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.561882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8058c-974d-4c2b-9b10-02f089f2859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.562665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da356ee7-4681-477d-a7e9-66a8ee525779 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.562946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8058c-974d-4c2b-9b10-02f089f2859b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.563328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8058c-974d-4c2b-9b10-02f089f2859b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.563671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8058c-974d-4c2b-9b10-02f089f2859b admin admin] PATH_INFO: `/v3/projects/d5d44767e53448818c9f747c83df0024/users/04753e4485014154b21f75e3dd948d60/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.565711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8058c-974d-4c2b-9b10-02f089f2859b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d44767e53448818c9f747c83df0024, user_id=04753e4485014154b21f75e3dd948d60, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.568093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.568457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8058c-974d-4c2b-9b10-02f089f2859b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.568754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.569033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.569307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.570061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:21:58.570365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.572011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.579307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1038/2054] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:58 2022] PUT /identity/v3/projects/d5d44767e53448818c9f747c83df0024/users/04753e4485014154b21f75e3dd948d60/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:58.590407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f81f2-a1fd-4ecc-86ad-31de01c1e9cb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.590816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f81f2-a1fd-4ecc-86ad-31de01c1e9cb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.591230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f81f2-a1fd-4ecc-86ad-31de01c1e9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.595821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1017/2055] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:21:58.597036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.606729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59659caf-6839-41da-b182-de80dc513999 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.619138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4b3f81f2-a1fd-4ecc-86ad-31de01c1e9cb None None] MFA Rules not processed for user `04753e4485014154b21f75e3dd948d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:58.620479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b3f81f2-a1fd-4ecc-86ad-31de01c1e9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:58.664973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.665625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.665991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.666184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.666900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.668838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.695097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1039/2056] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:21:58.702118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1018/2057] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:58 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:21:58.705261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f81f2-a1fd-4ecc-86ad-31de01c1e9cb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.706091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.708453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:58.725566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f81f2-a1fd-4ecc-86ad-31de01c1e9cb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.753263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:58.801142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91084-20a0-4972-895f-ec27b776f63b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.806614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.807308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.807592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.807889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.810469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:58.827085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d38ed-7a2b-4b86-b7c5-9dfe48f94c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.828565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d38ed-7a2b-4b86-b7c5-9dfe48f94c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.828565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d38ed-7a2b-4b86-b7c5-9dfe48f94c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.828565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d38ed-7a2b-4b86-b7c5-9dfe48f94c73 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.829984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d38ed-7a2b-4b86-b7c5-9dfe48f94c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.832040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d38ed-7a2b-4b86-b7c5-9dfe48f94c73 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.870278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1040/2058] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:58 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:58.871286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d38ed-7a2b-4b86-b7c5-9dfe48f94c73 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.881815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d38ed-7a2b-4b86-b7c5-9dfe48f94c73 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.886893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93db34-ef29-4159-8571-5b057096b773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.887597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93db34-ef29-4159-8571-5b057096b773 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.887881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93db34-ef29-4159-8571-5b057096b773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.888199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93db34-ef29-4159-8571-5b057096b773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.889057 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93db34-ef29-4159-8571-5b057096b773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.890928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93db34-ef29-4159-8571-5b057096b773 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.893073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:58.904560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.906357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.907248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1041/2059] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:58.911026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93db34-ef29-4159-8571-5b057096b773 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.919072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1019/2060] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:58.927034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93db34-ef29-4159-8571-5b057096b773 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.932767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0030b3d-7c93-4ebc-b740-977fc74c3e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.933436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0030b3d-7c93-4ebc-b740-977fc74c3e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.933716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0030b3d-7c93-4ebc-b740-977fc74c3e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.933990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0030b3d-7c93-4ebc-b740-977fc74c3e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.934726 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0030b3d-7c93-4ebc-b740-977fc74c3e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.936466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0030b3d-7c93-4ebc-b740-977fc74c3e8c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:58.946801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1042/2061] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:21:58.951249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:58.962630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1ccde-061c-4ddc-9c6c-aa2d25fe9286 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:58.967853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d32ee7-e61f-4137-bbbb-0b7856d5843d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:58.968864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d32ee7-e61f-4137-bbbb-0b7856d5843d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:58.969154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d32ee7-e61f-4137-bbbb-0b7856d5843d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:58.969431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d32ee7-e61f-4137-bbbb-0b7856d5843d admin admin] PATH_INFO: `/v3/projects/47ab0091359c44e29b433d13609d2305/users/3e4159fb2afe4023adc3d5fc257ea9fb/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:58.984994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d32ee7-e61f-4137-bbbb-0b7856d5843d admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ab0091359c44e29b433d13609d2305, user_id=3e4159fb2afe4023adc3d5fc257ea9fb, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:58.987781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d32ee7-e61f-4137-bbbb-0b7856d5843d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:59.002665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1020/2062] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:58 2022] PUT /identity/v3/projects/47ab0091359c44e29b433d13609d2305/users/3e4159fb2afe4023adc3d5fc257ea9fb/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:59.014034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee77e-32db-4b74-b037-60946130b64f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.014315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee77e-32db-4b74-b037-60946130b64f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.014589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee77e-32db-4b74-b037-60946130b64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.041146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ad8ee77e-32db-4b74-b037-60946130b64f None None] MFA Rules not processed for user `3e4159fb2afe4023adc3d5fc257ea9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:59.042370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ad8ee77e-32db-4b74-b037-60946130b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:59.111255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1043/2063] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:21:59.115691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d32ee7-e61f-4137-bbbb-0b7856d5843d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:59.196308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d32ee7-e61f-4137-bbbb-0b7856d5843d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:59.201708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063dff-ac7c-4a94-9218-d72ec523b801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:59.204523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063dff-ac7c-4a94-9218-d72ec523b801 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.204523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063dff-ac7c-4a94-9218-d72ec523b801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.204523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063dff-ac7c-4a94-9218-d72ec523b801 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.204523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4063dff-ac7c-4a94-9218-d72ec523b801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:59.206378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4063dff-ac7c-4a94-9218-d72ec523b801 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:59.243393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1021/2064] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:21:59 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:59.244721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ee77e-32db-4b74-b037-60946130b64f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:59.257014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ee77e-32db-4b74-b037-60946130b64f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:59.326969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce4c0-9398-43a4-92cb-218a47d49be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:59.327943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce4c0-9398-43a4-92cb-218a47d49be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.328370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce4c0-9398-43a4-92cb-218a47d49be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.328724 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce4c0-9398-43a4-92cb-218a47d49be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.329497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ce4c0-9398-43a4-92cb-218a47d49be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:59.331602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ce4c0-9398-43a4-92cb-218a47d49be4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:59.357764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1044/2065] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:21:59 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:21:59.359262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063dff-ac7c-4a94-9218-d72ec523b801 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:59.369562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063dff-ac7c-4a94-9218-d72ec523b801 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:59.375056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862c172-67aa-4f00-8cf5-aa92126b1adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:59.375875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862c172-67aa-4f00-8cf5-aa92126b1adf admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.376223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862c172-67aa-4f00-8cf5-aa92126b1adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.376593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862c172-67aa-4f00-8cf5-aa92126b1adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.377425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862c172-67aa-4f00-8cf5-aa92126b1adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:59.379188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862c172-67aa-4f00-8cf5-aa92126b1adf admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:59.391559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1022/2066] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:59.395822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce4c0-9398-43a4-92cb-218a47d49be4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:59.407484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce4c0-9398-43a4-92cb-218a47d49be4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:59.412729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b4e5b-f66a-4b90-a47e-04430b7e736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:59.413803 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b4e5b-f66a-4b90-a47e-04430b7e736c admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.414086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b4e5b-f66a-4b90-a47e-04430b7e736c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.414362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b4e5b-f66a-4b90-a47e-04430b7e736c admin admin] PATH_INFO: `/v3/projects/92253e4144074a949fd61fb4fd256ad9/users/a4b1eb46bab44438b4d1767056e4033c/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.433383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b4e5b-f66a-4b90-a47e-04430b7e736c admin admin] RBAC: Authorizing `identity:create_grant(project_id=92253e4144074a949fd61fb4fd256ad9, user_id=a4b1eb46bab44438b4d1767056e4033c, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:59.436464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b4e5b-f66a-4b90-a47e-04430b7e736c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:59.454581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1045/2067] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:59 2022] PUT /identity/v3/projects/92253e4144074a949fd61fb4fd256ad9/users/a4b1eb46bab44438b4d1767056e4033c/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:21:59.458789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862c172-67aa-4f00-8cf5-aa92126b1adf admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:59.469502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862c172-67aa-4f00-8cf5-aa92126b1adf admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:59.513978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:59.549565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd0ac6-040c-4b00-9a55-ff16dbe58137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:59.550281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0ac6-040c-4b00-9a55-ff16dbe58137 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.550575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0ac6-040c-4b00-9a55-ff16dbe58137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.550899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0ac6-040c-4b00-9a55-ff16dbe58137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.552114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd0ac6-040c-4b00-9a55-ff16dbe58137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:59.555978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd0ac6-040c-4b00-9a55-ff16dbe58137 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:59.571246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1023/2068] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:21:59.573077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b4e5b-f66a-4b90-a47e-04430b7e736c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:59.586462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b4e5b-f66a-4b90-a47e-04430b7e736c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:59.593997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1ef6-45b2-452f-9f01-8e25a8c46ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:59.594984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1ef6-45b2-452f-9f01-8e25a8c46ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.595357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1ef6-45b2-452f-9f01-8e25a8c46ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.595757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1ef6-45b2-452f-9f01-8e25a8c46ea6 admin admin] PATH_INFO: `/v3/projects/92253e4144074a949fd61fb4fd256ad9/users/a4b1eb46bab44438b4d1767056e4033c/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.602969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d1ef6-45b2-452f-9f01-8e25a8c46ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92253e4144074a949fd61fb4fd256ad9, user_id=a4b1eb46bab44438b4d1767056e4033c, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:59.605754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d1ef6-45b2-452f-9f01-8e25a8c46ea6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:59.619256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1046/2069] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:21:59 2022] PUT /identity/v3/projects/92253e4144074a949fd61fb4fd256ad9/users/a4b1eb46bab44438b4d1767056e4033c/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:21:59.628857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807badf-bc2a-46cf-8321-a57d7d5f7b9e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.629149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807badf-bc2a-46cf-8321-a57d7d5f7b9e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.629424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807badf-bc2a-46cf-8321-a57d7d5f7b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.661843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2807badf-bc2a-46cf-8321-a57d7d5f7b9e None None] MFA Rules not processed for user `a4b1eb46bab44438b4d1767056e4033c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:21:59.663097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2807badf-bc2a-46cf-8321-a57d7d5f7b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:59.739850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1024/2070] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:21:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:21:59.753880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1ef6-45b2-452f-9f01-8e25a8c46ea6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:59.784647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:21:59.811240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1ef6-45b2-452f-9f01-8e25a8c46ea6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:59.815032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70377a1-19d5-49cf-a6fa-04fc14929575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:59.815710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70377a1-19d5-49cf-a6fa-04fc14929575 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.815988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70377a1-19d5-49cf-a6fa-04fc14929575 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.816255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70377a1-19d5-49cf-a6fa-04fc14929575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:21:59.825665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70377a1-19d5-49cf-a6fa-04fc14929575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:21:59.827121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70377a1-19d5-49cf-a6fa-04fc14929575 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:21:59.837265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1047/2071] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:21:59.899258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807badf-bc2a-46cf-8321-a57d7d5f7b9e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:21:59.901003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2807badf-bc2a-46cf-8321-a57d7d5f7b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:21:59.913906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807badf-bc2a-46cf-8321-a57d7d5f7b9e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:21:59.917563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e8d1e-593e-4f8f-9202-c3fed0a0536d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:21:59.918222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e8d1e-593e-4f8f-9202-c3fed0a0536d service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:21:59.918508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e8d1e-593e-4f8f-9202-c3fed0a0536d service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:21:59.918787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e8d1e-593e-4f8f-9202-c3fed0a0536d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:00.005366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e8d1e-593e-4f8f-9202-c3fed0a0536d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:00.006833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e8d1e-593e-4f8f-9202-c3fed0a0536d service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:00.016129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1025/2072] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:21:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:00.462667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70377a1-19d5-49cf-a6fa-04fc14929575 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:00.472069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70377a1-19d5-49cf-a6fa-04fc14929575 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:00.475680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:00.476326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:00.476609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:00.476880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:00.478733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:00.546564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:00.548081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:00.559397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1048/2073] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:01.197604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e8d1e-593e-4f8f-9202-c3fed0a0536d service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:01.210116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e8d1e-593e-4f8f-9202-c3fed0a0536d service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:01.219262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347c9ce-0cd2-4cc7-808b-32cbb483899d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:01.219935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347c9ce-0cd2-4cc7-808b-32cbb483899d service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:01.220215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347c9ce-0cd2-4cc7-808b-32cbb483899d service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:01.220491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347c9ce-0cd2-4cc7-808b-32cbb483899d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:01.295768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:01.315130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347c9ce-0cd2-4cc7-808b-32cbb483899d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:01.319527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347c9ce-0cd2-4cc7-808b-32cbb483899d service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:01.331991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1026/2074] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:03.636870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:03.703054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc0fc9-227e-4399-9cbf-fb357c0402cd service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:03.707630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:03.708543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:03.708811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:03.709081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] PATH_INFO: `/v3/users/3e4159fb2afe4023adc3d5fc257ea9fb` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:03.710371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4159fb2afe4023adc3d5fc257ea9fb)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:03.712267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:03.765088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4159fb2afe4023adc3d5fc257ea9fb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:03.765756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4159fb2afe4023adc3d5fc257ea9fb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:03.772879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4159fb2afe4023adc3d5fc257ea9fb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:03.784313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:03.790689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4159fb2afe4023adc3d5fc257ea9fb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:03.802466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4159fb2afe4023adc3d5fc257ea9fb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:03.809845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1049/2075] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:03 2022] DELETE /identity/v3/users/3e4159fb2afe4023adc3d5fc257ea9fb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:03.814483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b6503-9fba-4a48-84f3-f05db20543e7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:03.815021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b6503-9fba-4a48-84f3-f05db20543e7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:03.815021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b6503-9fba-4a48-84f3-f05db20543e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:03.833750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-398b6503-9fba-4a48-84f3-f05db20543e7 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:03.834995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-398b6503-9fba-4a48-84f3-f05db20543e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:03.910232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1027/2076] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:03.921014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:03.924265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:04.021137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25e5c-6333-44bc-8708-37695420343e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:04.025299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7c89f-b357-472e-828c-a76bafe31f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:04.026263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7c89f-b357-472e-828c-a76bafe31f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:04.026559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7c89f-b357-472e-828c-a76bafe31f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:04.026835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7c89f-b357-472e-828c-a76bafe31f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:04.038684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7c89f-b357-472e-828c-a76bafe31f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:04.040216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7c89f-b357-472e-828c-a76bafe31f6a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:04.052297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1050/2077] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:22:04.412242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b6503-9fba-4a48-84f3-f05db20543e7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:04.414189 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-398b6503-9fba-4a48-84f3-f05db20543e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:04.426667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b6503-9fba-4a48-84f3-f05db20543e7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:04.432235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:04.435443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:04.441535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:04.442602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] PATH_INFO: `/v3/projects/47ab0091359c44e29b433d13609d2305` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:04.444651 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ab0091359c44e29b433d13609d2305)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:04.447401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:04.517273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ab0091359c44e29b433d13609d2305', 'request_id': 'req-95abaa68-d26e-48f3-b29f-83a267840889'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.523935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ab0091359c44e29b433d13609d2305', 'request_id': 'req-95abaa68-d26e-48f3-b29f-83a267840889'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.535902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ab0091359c44e29b433d13609d2305', 'request_id': 'req-95abaa68-d26e-48f3-b29f-83a267840889'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.550006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] The token cache is being invalidate because project 47ab0091359c44e29b433d13609d2305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:04.551375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.553986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.561127 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1028/2078] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:04 2022] DELETE /identity/v3/projects/47ab0091359c44e29b433d13609d2305 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:04.562527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7c89f-b357-472e-828c-a76bafe31f6a service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:04.585411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7c89f-b357-472e-828c-a76bafe31f6a service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:04.674660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:04.676431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:04.676949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:04.677328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] PATH_INFO: `/v3/users/a4b1eb46bab44438b4d1767056e4033c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:04.678933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b1eb46bab44438b4d1767056e4033c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:04.681005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:04.746364 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b1eb46bab44438b4d1767056e4033c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.747035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b1eb46bab44438b4d1767056e4033c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.761657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b1eb46bab44438b4d1767056e4033c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.779484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b1eb46bab44438b4d1767056e4033c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.796634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e8c83449-39cd-49e0-9fd4-7e361d3fa32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b1eb46bab44438b4d1767056e4033c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:04.807587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1051/2079] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:04 2022] DELETE /identity/v3/users/a4b1eb46bab44438b4d1767056e4033c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:05.230856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:05.232676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:05.249237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abaa68-d26e-48f3-b29f-83a267840889 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:05.313900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:05.315132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:05.315517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:05.316055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] PATH_INFO: `/v3/projects/92253e4144074a949fd61fb4fd256ad9` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:05.317351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92253e4144074a949fd61fb4fd256ad9)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:05.319318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:05.332264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:05.384879 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92253e4144074a949fd61fb4fd256ad9', 'request_id': 'req-8a4069f0-3ffb-459e-b082-12bb96219259'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.395177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92253e4144074a949fd61fb4fd256ad9', 'request_id': 'req-8a4069f0-3ffb-459e-b082-12bb96219259'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.405000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92253e4144074a949fd61fb4fd256ad9', 'request_id': 'req-8a4069f0-3ffb-459e-b082-12bb96219259'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.418294 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] The token cache is being invalidate because project 92253e4144074a949fd61fb4fd256ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:05.418758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.419636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.421374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1029/2080] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:05 2022] DELETE /identity/v3/projects/92253e4144074a949fd61fb4fd256ad9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:05.427783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427727-9426-4b07-b3a8-1150af04d786 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:05.428073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427727-9426-4b07-b3a8-1150af04d786 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:05.428354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427727-9426-4b07-b3a8-1150af04d786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:05.443606 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:05.445126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:05.449941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6b427727-9426-4b07-b3a8-1150af04d786 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:05.451747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6b427727-9426-4b07-b3a8-1150af04d786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:05.457906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:05.556461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4069f0-3ffb-459e-b082-12bb96219259 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:05.556664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1052/2081] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:05.562057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:05.562966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:05.563572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:05.563937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] PATH_INFO: `/v3/users/9a6df2f12caa4c99806ef20a6b91ffc7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:05.565302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6df2f12caa4c99806ef20a6b91ffc7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:05.565573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15dd45-93ba-435e-8de2-98ffcd25fd54 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:05.565878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15dd45-93ba-435e-8de2-98ffcd25fd54 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:05.566169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15dd45-93ba-435e-8de2-98ffcd25fd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:05.567320 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:05.595439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2e15dd45-93ba-435e-8de2-98ffcd25fd54 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:05.596918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2e15dd45-93ba-435e-8de2-98ffcd25fd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:05.610347 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1053/2082] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:22:05.616535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15dd45-93ba-435e-8de2-98ffcd25fd54 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:05.630611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6df2f12caa4c99806ef20a6b91ffc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.638263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15dd45-93ba-435e-8de2-98ffcd25fd54 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:05.644173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6df2f12caa4c99806ef20a6b91ffc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.653790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:05.654872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:05.656215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:05.657744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:05.658500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:22:05.659520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:05.663855 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:05.670735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6df2f12caa4c99806ef20a6b91ffc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.676432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6df2f12caa4c99806ef20a6b91ffc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.689445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1054/2083] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:22:05.693899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:05.694188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cf56decd-cc63-42f5-bfd3-78e9643f5b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6df2f12caa4c99806ef20a6b91ffc7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:05.700797 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1030/2084] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:05 2022] DELETE /identity/v3/users/9a6df2f12caa4c99806ef20a6b91ffc7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:05.704493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f56813-9891-43ae-9f81-8b3903060701 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:05.705018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952bfa3-1a1e-481b-83e8-2d928690775b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:05.705399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952bfa3-1a1e-481b-83e8-2d928690775b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:05.705764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952bfa3-1a1e-481b-83e8-2d928690775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:05.725053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2952bfa3-1a1e-481b-83e8-2d928690775b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:05.726433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2952bfa3-1a1e-481b-83e8-2d928690775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:05.742464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:05.771661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:05.773866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:05.774878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:05.775241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:05.778481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:05.793887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:05.833991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1031/2085] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:05.859557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952bfa3-1a1e-481b-83e8-2d928690775b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:05.866891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2952bfa3-1a1e-481b-83e8-2d928690775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:05.879378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1055/2086] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:05 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:05.881599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:05.883385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:05.969117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fc534-d663-4c3f-ae01-e82fe65271ec admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:05.977812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16231fa-4ba8-455e-b522-310d3c071f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:05.978716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16231fa-4ba8-455e-b522-310d3c071f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:05.978998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16231fa-4ba8-455e-b522-310d3c071f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:05.979619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16231fa-4ba8-455e-b522-310d3c071f64 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:05.980076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16231fa-4ba8-455e-b522-310d3c071f64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:05.982956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16231fa-4ba8-455e-b522-310d3c071f64 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.026026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952bfa3-1a1e-481b-83e8-2d928690775b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.054182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1056/2087] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:05 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:22:06.055123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16231fa-4ba8-455e-b522-310d3c071f64 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:06.068765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:06.075249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16231fa-4ba8-455e-b522-310d3c071f64 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.086998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaa62d-a4ec-452e-bd17-c46fd462eb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.088130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaa62d-a4ec-452e-bd17-c46fd462eb5a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.088590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaa62d-a4ec-452e-bd17-c46fd462eb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.089327 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaa62d-a4ec-452e-bd17-c46fd462eb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.090296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaaa62d-a4ec-452e-bd17-c46fd462eb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:06.103858 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaaa62d-a4ec-452e-bd17-c46fd462eb5a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.124480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1057/2088] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:06.129246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaa62d-a4ec-452e-bd17-c46fd462eb5a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:06.134322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7a7b-f302-4947-9043-1dcde4403408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.135264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7a7b-f302-4947-9043-1dcde4403408 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.136310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7a7b-f302-4947-9043-1dcde4403408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.136310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7a7b-f302-4947-9043-1dcde4403408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.150958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaa62d-a4ec-452e-bd17-c46fd462eb5a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.154021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba7a7b-f302-4947-9043-1dcde4403408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:06.155453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba7a7b-f302-4947-9043-1dcde4403408 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.161771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283148b5-29dc-49a2-a8d4-d8efb0410a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.162580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283148b5-29dc-49a2-a8d4-d8efb0410a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.162862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283148b5-29dc-49a2-a8d4-d8efb0410a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.163256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283148b5-29dc-49a2-a8d4-d8efb0410a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.166221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283148b5-29dc-49a2-a8d4-d8efb0410a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:06.168624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283148b5-29dc-49a2-a8d4-d8efb0410a4c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.173279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1032/2089] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:06.203673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1058/2090] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:06.211898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7a7b-f302-4947-9043-1dcde4403408 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:06.247085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7a7b-f302-4947-9043-1dcde4403408 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.261984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777b9c7-133a-48ee-b8ac-d686e31e0b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.266155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777b9c7-133a-48ee-b8ac-d686e31e0b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.274036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777b9c7-133a-48ee-b8ac-d686e31e0b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.274422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777b9c7-133a-48ee-b8ac-d686e31e0b96 admin admin] PATH_INFO: `/v3/projects/5603918ebd614f29bafc91c5697cfd10/users/0e1227721c0e4da891e29e90de2a5a1c/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.317031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777b9c7-133a-48ee-b8ac-d686e31e0b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5603918ebd614f29bafc91c5697cfd10, user_id=0e1227721c0e4da891e29e90de2a5a1c, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:06.321039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777b9c7-133a-48ee-b8ac-d686e31e0b96 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.346219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1033/2091] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:06 2022] PUT /identity/v3/projects/5603918ebd614f29bafc91c5697cfd10/users/0e1227721c0e4da891e29e90de2a5a1c/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:06.358910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c98d-0470-4cb6-9fe9-8dcc33b733b4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.359313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c98d-0470-4cb6-9fe9-8dcc33b733b4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.359907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c98d-0470-4cb6-9fe9-8dcc33b733b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.414520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fcf4c98d-0470-4cb6-9fe9-8dcc33b733b4 None None] MFA Rules not processed for user `0e1227721c0e4da891e29e90de2a5a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:06.416083 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fcf4c98d-0470-4cb6-9fe9-8dcc33b733b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:06.532073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1059/2092] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:06.536270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777b9c7-133a-48ee-b8ac-d686e31e0b96 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:06.627815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:06.638548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777b9c7-133a-48ee-b8ac-d686e31e0b96 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.650247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0f038-a12f-472d-b60d-79b12d19a852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.650987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0f038-a12f-472d-b60d-79b12d19a852 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.651317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0f038-a12f-472d-b60d-79b12d19a852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.651592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0f038-a12f-472d-b60d-79b12d19a852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.652303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0f038-a12f-472d-b60d-79b12d19a852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:06.654607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0f038-a12f-472d-b60d-79b12d19a852 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.686503 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1034/2093] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:06 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:06.691109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c98d-0470-4cb6-9fe9-8dcc33b733b4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:06.712475 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c98d-0470-4cb6-9fe9-8dcc33b733b4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.832024 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.833028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.833470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.833916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.837307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:06.840132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.883003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1060/2094] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:06 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:22:06.887113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0f038-a12f-472d-b60d-79b12d19a852 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:06.887633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:06.890474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:06.899520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0f038-a12f-472d-b60d-79b12d19a852 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.906300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4090-bf45-44c8-9195-841b6f8b77ea admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.911815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69026e-1b48-4310-81ef-04d3469cbeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.913252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.913382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69026e-1b48-4310-81ef-04d3469cbeae admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.913890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69026e-1b48-4310-81ef-04d3469cbeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.914135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.914420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.914699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] PATH_INFO: `/v3/projects/634bcb327f294ad08e43faf2c3d21d64` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.915468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69026e-1b48-4310-81ef-04d3469cbeae admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:06.916382 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69026e-1b48-4310-81ef-04d3469cbeae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:06.916611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=634bcb327f294ad08e43faf2c3d21d64)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:06.918922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69026e-1b48-4310-81ef-04d3469cbeae admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.919420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:06.943093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1035/2095] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:22:06.947192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69026e-1b48-4310-81ef-04d3469cbeae admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:06.964381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69026e-1b48-4310-81ef-04d3469cbeae admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:06.971526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:06.973196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:06.973629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:06.974720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] PATH_INFO: `/v3/projects/fae74f3c8aea4a6ca89f867f7665d4f0/users/254e3ba112c3460fa32c283628a47ce6/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.030915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '634bcb327f294ad08e43faf2c3d21d64', 'request_id': 'req-9e788194-8b91-4430-9acf-1300ae86ab62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.034033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae74f3c8aea4a6ca89f867f7665d4f0, user_id=254e3ba112c3460fa32c283628a47ce6, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:07.037101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:07.050683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:07.052934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '634bcb327f294ad08e43faf2c3d21d64', 'request_id': 'req-9e788194-8b91-4430-9acf-1300ae86ab62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.063375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '634bcb327f294ad08e43faf2c3d21d64', 'request_id': 'req-9e788194-8b91-4430-9acf-1300ae86ab62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.063882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1036/2096] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:06 2022] PUT /identity/v3/projects/fae74f3c8aea4a6ca89f867f7665d4f0/users/254e3ba112c3460fa32c283628a47ce6/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:07.065515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:07.067748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.080562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] The token cache is being invalidate because project 634bcb327f294ad08e43faf2c3d21d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:07.080945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.081665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.083591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1061/2097] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:06 2022] DELETE /identity/v3/projects/634bcb327f294ad08e43faf2c3d21d64 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:07.084619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:07.087747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.159303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66722e8-c896-41c7-ae8c-7675d1197f76 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:07.168277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:07.169182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.169521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.169792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] PATH_INFO: `/v3/users/45b8a18312134372a867a578f1cf55f3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.171092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b8a18312134372a867a578f1cf55f3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:07.173071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:07.188163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788194-8b91-4430-9acf-1300ae86ab62 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:07.193790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:07.194553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.194902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.195269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.196317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:07.198005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:07.214098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1062/2098] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:07.214098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:07.214098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.226751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38541e14-e543-47d8-bad2-cee51626de7d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:07.231032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b8a18312134372a867a578f1cf55f3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.232002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:07.232914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.233308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.233657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] PATH_INFO: `/v3/users/e1500a9020714f5994d9adabed528f28` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.234945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1500a9020714f5994d9adabed528f28)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:07.237132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:07.240533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b8a18312134372a867a578f1cf55f3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.248907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b8a18312134372a867a578f1cf55f3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.249741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b8a18312134372a867a578f1cf55f3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.263125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b8a18312134372a867a578f1cf55f3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.271706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1037/2099] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:07 2022] DELETE /identity/v3/users/45b8a18312134372a867a578f1cf55f3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:07.272991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:07.274981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.315305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:07.326714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1500a9020714f5994d9adabed528f28'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.327400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1500a9020714f5994d9adabed528f28'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.337175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1500a9020714f5994d9adabed528f28'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.356116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1500a9020714f5994d9adabed528f28'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.364934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1500a9020714f5994d9adabed528f28'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.374489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1063/2100] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:07 2022] DELETE /identity/v3/users/e1500a9020714f5994d9adabed528f28 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:07.377692 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c673e6-96b6-4df4-9072-d4407e097012 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:07.391258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:07.391258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.502812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:07.503814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.504096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.504572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] PATH_INFO: `/v3/users/eaf926af634c4ec2ada5fc8f4aa95e07` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.505259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:07.508341 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf926af634c4ec2ada5fc8f4aa95e07)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:07.511986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:07.532998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8a63-d721-40f4-9e4d-1fd7982f12f4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:07.539257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ac3e4-80d5-4e2e-9b63-ad1800050746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:07.540283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac3e4-80d5-4e2e-9b63-ad1800050746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.540571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac3e4-80d5-4e2e-9b63-ad1800050746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.540842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac3e4-80d5-4e2e-9b63-ad1800050746 admin admin] PATH_INFO: `/v3/projects/fae74f3c8aea4a6ca89f867f7665d4f0/users/254e3ba112c3460fa32c283628a47ce6/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.543985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ac3e4-80d5-4e2e-9b63-ad1800050746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae74f3c8aea4a6ca89f867f7665d4f0, user_id=254e3ba112c3460fa32c283628a47ce6, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:07.548110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ac3e4-80d5-4e2e-9b63-ad1800050746 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:07.570305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1064/2101] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:07 2022] PUT /identity/v3/projects/fae74f3c8aea4a6ca89f867f7665d4f0/users/254e3ba112c3460fa32c283628a47ce6/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:07.571734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff43b-5575-453f-8e29-ea7e8f25c823 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.572159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff43b-5575-453f-8e29-ea7e8f25c823 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.572444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff43b-5575-453f-8e29-ea7e8f25c823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.606027 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b4aff43b-5575-453f-8e29-ea7e8f25c823 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:07.608663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b4aff43b-5575-453f-8e29-ea7e8f25c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.614494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf926af634c4ec2ada5fc8f4aa95e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.628699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf926af634c4ec2ada5fc8f4aa95e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.643263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf926af634c4ec2ada5fc8f4aa95e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.644112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf926af634c4ec2ada5fc8f4aa95e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.672079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b59ece41-03a0-419b-8233-7bdce3ae0543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf926af634c4ec2ada5fc8f4aa95e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:07.688165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1038/2102] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:07 2022] DELETE /identity/v3/users/eaf926af634c4ec2ada5fc8f4aa95e07 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:07.689932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f821d63-b432-4cd1-b5ba-75367ce70ee9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.690229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f821d63-b432-4cd1-b5ba-75367ce70ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.690509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f821d63-b432-4cd1-b5ba-75367ce70ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.733802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1065/2103] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:07.744910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6f821d63-b432-4cd1-b5ba-75367ce70ee9 None None] MFA Rules not processed for user `254e3ba112c3460fa32c283628a47ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:07.746297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6f821d63-b432-4cd1-b5ba-75367ce70ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.754533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aff43b-5575-453f-8e29-ea7e8f25c823 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:07.763458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b4aff43b-5575-453f-8e29-ea7e8f25c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.798792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:07.840058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aff43b-5575-453f-8e29-ea7e8f25c823 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:07.847656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1039/2104] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:07.849010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f821d63-b432-4cd1-b5ba-75367ce70ee9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:07.856928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6f821d63-b432-4cd1-b5ba-75367ce70ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:07.947739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f821d63-b432-4cd1-b5ba-75367ce70ee9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:07.947977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:07.948711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.948997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.949366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.953236 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:07.954238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:07.954624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:07.955000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] PATH_INFO: `/v3/projects/c60bde0b6ac94b17bcc2ce5da133bbd7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:07.956376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60bde0b6ac94b17bcc2ce5da133bbd7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:07.958515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:07.960970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:07.962512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:07.974355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1066/2105] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:22:07.975049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:07.976372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.014760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60bde0b6ac94b17bcc2ce5da133bbd7', 'request_id': 'req-a6c0075d-2784-4964-aa21-20e1ded0c1b0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.021635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60bde0b6ac94b17bcc2ce5da133bbd7', 'request_id': 'req-a6c0075d-2784-4964-aa21-20e1ded0c1b0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.032942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60bde0b6ac94b17bcc2ce5da133bbd7', 'request_id': 'req-a6c0075d-2784-4964-aa21-20e1ded0c1b0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.044294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:08.052025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] The token cache is being invalidate because project c60bde0b6ac94b17bcc2ce5da133bbd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:08.052745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.053528 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c822fd-03b6-494f-8100-e7e1a6d778cb service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:08.054603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.058561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1040/2106] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:07 2022] DELETE /identity/v3/projects/c60bde0b6ac94b17bcc2ce5da133bbd7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:08.063631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:08.065617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.150505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:08.151352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.151693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.152230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.154556 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.200311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0075d-2784-4964-aa21-20e1ded0c1b0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:08.209504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:08.210443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.210725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.210998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] PATH_INFO: `/v3/users/04753e4485014154b21f75e3dd948d60` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.212332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04753e4485014154b21f75e3dd948d60)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:08.214430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:08.223035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:08.265374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:08.266995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:08.274248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04753e4485014154b21f75e3dd948d60'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.278805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1067/2107] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:08.289718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:08.296095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.300126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04753e4485014154b21f75e3dd948d60'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.318973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04753e4485014154b21f75e3dd948d60'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.319908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04753e4485014154b21f75e3dd948d60'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.326413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e29ac-ee3b-4f7c-8929-8d53717e8e3e service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:08.332691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:08.333782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.334476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.334851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] PATH_INFO: `/v3/projects/0e2534820f9840f5b45a08bb2428be70` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.336843 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2534820f9840f5b45a08bb2428be70)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:08.339036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:08.342438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04753e4485014154b21f75e3dd948d60'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.351632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1041/2108] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:08 2022] DELETE /identity/v3/users/04753e4485014154b21f75e3dd948d60 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:08.360239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:08.361671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.424814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2534820f9840f5b45a08bb2428be70', 'request_id': 'req-7277e7be-fe96-4736-8f14-87f26ff8fa35'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.430283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2534820f9840f5b45a08bb2428be70', 'request_id': 'req-7277e7be-fe96-4736-8f14-87f26ff8fa35'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.440628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2534820f9840f5b45a08bb2428be70', 'request_id': 'req-7277e7be-fe96-4736-8f14-87f26ff8fa35'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.449952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8fdc2-1e70-4f4a-bdb9-942e71c52286 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:08.454642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] The token cache is being invalidate because project 0e2534820f9840f5b45a08bb2428be70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:08.455054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.455847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.457535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1068/2109] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:08 2022] DELETE /identity/v3/projects/0e2534820f9840f5b45a08bb2428be70 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:08.458578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:08.460056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.510481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:08.530833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:08.532204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.532620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.532961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] PATH_INFO: `/v3/projects/b574edec2fec48a98bc1c31cfc58afaa` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.534375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b574edec2fec48a98bc1c31cfc58afaa)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:08.536538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:08.539481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e7be-fe96-4736-8f14-87f26ff8fa35 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:08.543189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:08.543874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.544153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.544523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.546533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.596114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:08.611468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b574edec2fec48a98bc1c31cfc58afaa', 'request_id': 'req-c8ec242f-ebe4-487c-be0f-32b6970cc17c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.621218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b574edec2fec48a98bc1c31cfc58afaa', 'request_id': 'req-c8ec242f-ebe4-487c-be0f-32b6970cc17c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.631966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b574edec2fec48a98bc1c31cfc58afaa', 'request_id': 'req-c8ec242f-ebe4-487c-be0f-32b6970cc17c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.650824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] The token cache is being invalidate because project b574edec2fec48a98bc1c31cfc58afaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:08.651161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.651946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c8ec242f-ebe4-487c-be0f-32b6970cc17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:08.653143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:08.654753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:08.657507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1042/2110] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:08 2022] DELETE /identity/v3/projects/b574edec2fec48a98bc1c31cfc58afaa => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:08.659642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470b52f-769c-4269-870b-6891180868e2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.660929 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470b52f-769c-4269-870b-6891180868e2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.661452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470b52f-769c-4269-870b-6891180868e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.666486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1069/2111] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:08.690923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a470b52f-769c-4269-870b-6891180868e2 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:08.694105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a470b52f-769c-4269-870b-6891180868e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.765054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:08.766538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.792117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dfeeb-2c57-4b5c-a2dc-00d76382a578 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:08.871549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1043/2112] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:08.881448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9eb7-87ab-4320-b203-05ac25abf0ad None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.882468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9eb7-87ab-4320-b203-05ac25abf0ad None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.882791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9eb7-87ab-4320-b203-05ac25abf0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.921303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a64d9eb7-87ab-4320-b203-05ac25abf0ad None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:08.922560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a64d9eb7-87ab-4320-b203-05ac25abf0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:08.932260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1044/2113] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:08.936248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:08.936599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d9eb7-87ab-4320-b203-05ac25abf0ad None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:08.937593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.937899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.938172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] PATH_INFO: `/v3/projects/d5d44767e53448818c9f747c83df0024` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.939569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d44767e53448818c9f747c83df0024)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:08.941899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:08.946429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d9eb7-87ab-4320-b203-05ac25abf0ad None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:08.951763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:08.952511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:08.952947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:08.953199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:08.953986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:22:08.954354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:08.956070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:08.969159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1045/2114] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:22:08.974023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:08.986284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195bfad-295f-4c8d-b50d-8cd8615356c2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:08.998692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d44767e53448818c9f747c83df0024', 'request_id': 'req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.004899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d44767e53448818c9f747c83df0024', 'request_id': 'req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.013978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d44767e53448818c9f747c83df0024', 'request_id': 'req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.027496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] The token cache is being invalidate because project d5d44767e53448818c9f747c83df0024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:09.027782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.028885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-374de82b-7c18-4b50-9523-ae9e3d6f2b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.030632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1070/2115] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:08 2022] DELETE /identity/v3/projects/d5d44767e53448818c9f747c83df0024 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:09.035268 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d8b53-a77a-47be-a95f-989049270ca1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.035538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d8b53-a77a-47be-a95f-989049270ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.035818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d8b53-a77a-47be-a95f-989049270ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.055944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-643d8b53-a77a-47be-a95f-989049270ca1 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:09.057146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-643d8b53-a77a-47be-a95f-989049270ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.062275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.063013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.063410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.063742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.064567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.066396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.066922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1071/2116] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:22:09.074297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8cdef-0531-4096-b6c5-0a5c6885991f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.074587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8cdef-0531-4096-b6c5-0a5c6885991f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.074864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8cdef-0531-4096-b6c5-0a5c6885991f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.092766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e0f8cdef-0531-4096-b6c5-0a5c6885991f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:09.093864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e0f8cdef-0531-4096-b6c5-0a5c6885991f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.096237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1046/2117] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:08 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:09.100445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.102037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.102995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:09.134287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:09.159583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1072/2118] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:22:09.161613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8cdef-0531-4096-b6c5-0a5c6885991f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.163740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e0f8cdef-0531-4096-b6c5-0a5c6885991f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.177056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87ca14-ba8c-4fc7-8b1d-044efdbd9408 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.178840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8cdef-0531-4096-b6c5-0a5c6885991f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.183461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.184323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd97d3-045a-40fe-aeb9-a37f38bc4b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.184560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.184949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.185110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd97d3-045a-40fe-aeb9-a37f38bc4b93 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.185322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd97d3-045a-40fe-aeb9-a37f38bc4b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.185422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] PATH_INFO: `/v3/users/0e1227721c0e4da891e29e90de2a5a1c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.185591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd97d3-045a-40fe-aeb9-a37f38bc4b93 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.186282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd97d3-045a-40fe-aeb9-a37f38bc4b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.186772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1227721c0e4da891e29e90de2a5a1c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.188269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd97d3-045a-40fe-aeb9-a37f38bc4b93 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.188870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.209064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1073/2119] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:09.210000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd97d3-045a-40fe-aeb9-a37f38bc4b93 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.220572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd97d3-045a-40fe-aeb9-a37f38bc4b93 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.225565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.226216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.226498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.226772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.227538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:22:09.227831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.229431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.237567 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1227721c0e4da891e29e90de2a5a1c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.241695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1074/2120] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:22:09.242492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.250218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1227721c0e4da891e29e90de2a5a1c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.252541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36d05d-f1d0-49bc-b8c2-dfafcfd5f6bd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.257103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.257675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1227721c0e4da891e29e90de2a5a1c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.257791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.258037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.258307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.258523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1227721c0e4da891e29e90de2a5a1c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.259059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.260671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.270694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1075/2121] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:09.271439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1227721c0e4da891e29e90de2a5a1c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:09.271568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.275033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.277656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1047/2122] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:09 2022] DELETE /identity/v3/users/0e1227721c0e4da891e29e90de2a5a1c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:09.278481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.280625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.289566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0d206-f54f-491a-baaf-c976aef0747f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.345324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbfb3c-0cac-4881-bdaa-424a2f5b4947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.345892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfb3c-0cac-4881-bdaa-424a2f5b4947 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.346039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d63a3d-79fd-489a-9e85-bd0f6f1e9ead admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.346139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfb3c-0cac-4881-bdaa-424a2f5b4947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.346425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfb3c-0cac-4881-bdaa-424a2f5b4947 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.347086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbfb3c-0cac-4881-bdaa-424a2f5b4947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.349072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbfb3c-0cac-4881-bdaa-424a2f5b4947 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.350697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591ca89-59e2-4cc4-b584-e4ff48534b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.351388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ca89-59e2-4cc4-b584-e4ff48534b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.351669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ca89-59e2-4cc4-b584-e4ff48534b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.351946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ca89-59e2-4cc4-b584-e4ff48534b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.352667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591ca89-59e2-4cc4-b584-e4ff48534b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.354250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591ca89-59e2-4cc4-b584-e4ff48534b58 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.364825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1048/2123] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:09.371528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40c13b-0de4-4889-b7e4-ba5e9ae670d4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.371528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40c13b-0de4-4889-b7e4-ba5e9ae670d4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.371528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40c13b-0de4-4889-b7e4-ba5e9ae670d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.383258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1076/2124] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:09.384068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbfb3c-0cac-4881-bdaa-424a2f5b4947 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.385971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3d40c13b-0de4-4889-b7e4-ba5e9ae670d4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:09.387152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3d40c13b-0de4-4889-b7e4-ba5e9ae670d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.429612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:09.443063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:09.447372 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbfb3c-0cac-4881-bdaa-424a2f5b4947 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.453253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1049/2125] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:09.456492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.456492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.456721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40c13b-0de4-4889-b7e4-ba5e9ae670d4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.456804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.456804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] PATH_INFO: `/v3/projects/6f2b5f47c2234f578b4567862cdde573/users/96baad43289d4a6bb30bb5e6cf396e25/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.471940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40c13b-0de4-4889-b7e4-ba5e9ae670d4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.479726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153cbea-b1c6-4f5b-8d84-e63562812338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.480422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cbea-b1c6-4f5b-8d84-e63562812338 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.480630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2b5f47c2234f578b4567862cdde573, user_id=96baad43289d4a6bb30bb5e6cf396e25, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.480800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cbea-b1c6-4f5b-8d84-e63562812338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.480956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cbea-b1c6-4f5b-8d84-e63562812338 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.481680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153cbea-b1c6-4f5b-8d84-e63562812338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.484221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.484348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153cbea-b1c6-4f5b-8d84-e63562812338 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.503859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1077/2126] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:09 2022] PUT /identity/v3/projects/6f2b5f47c2234f578b4567862cdde573/users/96baad43289d4a6bb30bb5e6cf396e25/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:09.505056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.510482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.519516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1050/2127] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/users => generated 286 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:09.520888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b124ff-d139-4fdc-8789-31ffbe883e47 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.521177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b124ff-d139-4fdc-8789-31ffbe883e47 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.521448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b124ff-d139-4fdc-8789-31ffbe883e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.549244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-67b124ff-d139-4fdc-8789-31ffbe883e47 None None] MFA Rules not processed for user `96baad43289d4a6bb30bb5e6cf396e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:09.550438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-67b124ff-d139-4fdc-8789-31ffbe883e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.569582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a1368-e4e0-4a45-8df6-791966c71316 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.573165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a43fe9-28c4-4772-9c19-3efdeab99908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.573808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a43fe9-28c4-4772-9c19-3efdeab99908 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.574089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a43fe9-28c4-4772-9c19-3efdeab99908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.574363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a43fe9-28c4-4772-9c19-3efdeab99908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.616451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1051/2128] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:09.620332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b124ff-d139-4fdc-8789-31ffbe883e47 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.640866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a43fe9-28c4-4772-9c19-3efdeab99908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.642322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a43fe9-28c4-4772-9c19-3efdeab99908 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.652659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1078/2129] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:09.653507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a43fe9-28c4-4772-9c19-3efdeab99908 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.663094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a43fe9-28c4-4772-9c19-3efdeab99908 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.668291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cebeb-dcf0-4794-8aa8-35245a0d07e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.668935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cebeb-dcf0-4794-8aa8-35245a0d07e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.669346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cebeb-dcf0-4794-8aa8-35245a0d07e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.669620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cebeb-dcf0-4794-8aa8-35245a0d07e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.670298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cebeb-dcf0-4794-8aa8-35245a0d07e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.672373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cebeb-dcf0-4794-8aa8-35245a0d07e6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.679168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b124ff-d139-4fdc-8789-31ffbe883e47 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.684593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee2c0-8668-4c4c-8019-7c2f0cc5b607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.685358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee2c0-8668-4c4c-8019-7c2f0cc5b607 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.685648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee2c0-8668-4c4c-8019-7c2f0cc5b607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.685944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee2c0-8668-4c4c-8019-7c2f0cc5b607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.686678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ee2c0-8668-4c4c-8019-7c2f0cc5b607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.688343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ee2c0-8668-4c4c-8019-7c2f0cc5b607 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.700150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1052/2130] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:09.701549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1079/2131] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:09.706856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee2c0-8668-4c4c-8019-7c2f0cc5b607 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.707051 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cebeb-dcf0-4794-8aa8-35245a0d07e6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.717949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee2c0-8668-4c4c-8019-7c2f0cc5b607 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.776300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:09.781597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cebeb-dcf0-4794-8aa8-35245a0d07e6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.791153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c7f16-fe4b-4a3c-82aa-52c8b7f51504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.791684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daea89f-4309-4076-bfc3-fee181f2ff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.791684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea89f-4309-4076-bfc3-fee181f2ff88 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.791684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea89f-4309-4076-bfc3-fee181f2ff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.791684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea89f-4309-4076-bfc3-fee181f2ff88 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.792014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daea89f-4309-4076-bfc3-fee181f2ff88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.792102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c7f16-fe4b-4a3c-82aa-52c8b7f51504 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.792433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c7f16-fe4b-4a3c-82aa-52c8b7f51504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.792682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c7f16-fe4b-4a3c-82aa-52c8b7f51504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.793699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c7f16-fe4b-4a3c-82aa-52c8b7f51504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.794958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daea89f-4309-4076-bfc3-fee181f2ff88 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.799132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c7f16-fe4b-4a3c-82aa-52c8b7f51504 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.819969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1080/2132] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:09.825260 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c7f16-fe4b-4a3c-82aa-52c8b7f51504 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.832804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:09.837032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c7f16-fe4b-4a3c-82aa-52c8b7f51504 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.837182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1053/2133] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:09.841524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daea89f-4309-4076-bfc3-fee181f2ff88 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.846140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b08e4-bc3f-4257-ae30-f033c1173d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.847311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b08e4-bc3f-4257-ae30-f033c1173d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.847658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b08e4-bc3f-4257-ae30-f033c1173d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.848548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b08e4-bc3f-4257-ae30-f033c1173d61 admin admin] PATH_INFO: `/v3/projects/ad9e87cb4bc84ff294c69c2e54704c33/users/2821263e860a449eb4472ed2c4203e20/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.852672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daea89f-4309-4076-bfc3-fee181f2ff88 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.856755 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f1b55-2de4-49f6-99d1-64e4c53ca2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.857843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f1b55-2de4-49f6-99d1-64e4c53ca2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.858207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f1b55-2de4-49f6-99d1-64e4c53ca2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.858552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f1b55-2de4-49f6-99d1-64e4c53ca2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.859327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f1b55-2de4-49f6-99d1-64e4c53ca2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.862478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f1b55-2de4-49f6-99d1-64e4c53ca2d3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.865674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b08e4-bc3f-4257-ae30-f033c1173d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9e87cb4bc84ff294c69c2e54704c33, user_id=2821263e860a449eb4472ed2c4203e20, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.868746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b08e4-bc3f-4257-ae30-f033c1173d61 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.875029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1054/2134] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:09.878970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f1b55-2de4-49f6-99d1-64e4c53ca2d3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.889110 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1081/2135] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:09 2022] PUT /identity/v3/projects/ad9e87cb4bc84ff294c69c2e54704c33/users/2821263e860a449eb4472ed2c4203e20/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:09.890542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267d81-1f11-457b-8037-a0d183b9936c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.890830 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267d81-1f11-457b-8037-a0d183b9936c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.891281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267d81-1f11-457b-8037-a0d183b9936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.892734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f1b55-2de4-49f6-99d1-64e4c53ca2d3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:09.918057 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-57267d81-1f11-457b-8037-a0d183b9936c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:09.919608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-57267d81-1f11-457b-8037-a0d183b9936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:09.961400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:09.962312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:09.962712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:09.963044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] PATH_INFO: `/v3/projects/baff0c96256e4029a829875c2498ad83/users/a867de9112c3449a92861c8a30813814/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:09.981350 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baff0c96256e4029a829875c2498ad83, user_id=a867de9112c3449a92861c8a30813814, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:09.984582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:09.996771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1082/2136] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:09.996771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57267d81-1f11-457b-8037-a0d183b9936c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:09.996771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-57267d81-1f11-457b-8037-a0d183b9936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:10.012153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57267d81-1f11-457b-8037-a0d183b9936c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:10.019535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1055/2137] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:09 2022] PUT /identity/v3/projects/baff0c96256e4029a829875c2498ad83/users/a867de9112c3449a92861c8a30813814/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:10.022851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:10.022851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:10.086716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:10.087677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.088103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.088380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] PATH_INFO: `/v3/projects/5603918ebd614f29bafc91c5697cfd10` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.089620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5603918ebd614f29bafc91c5697cfd10)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:10.089984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb187c-c114-4da9-bca8-b3f9a5a46753 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:10.091722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:10.094136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b4cab-8607-41d4-9d1b-ff9a5af972b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:10.094905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b4cab-8607-41d4-9d1b-ff9a5af972b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.095322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b4cab-8607-41d4-9d1b-ff9a5af972b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.095698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b4cab-8607-41d4-9d1b-ff9a5af972b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.106426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b4cab-8607-41d4-9d1b-ff9a5af972b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:10.108020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b4cab-8607-41d4-9d1b-ff9a5af972b0 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:10.140551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1056/2138] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:10.141652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b4cab-8607-41d4-9d1b-ff9a5af972b0 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:10.152037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b4cab-8607-41d4-9d1b-ff9a5af972b0 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:10.160098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5603918ebd614f29bafc91c5697cfd10', 'request_id': 'req-c89634f5-4c36-47bb-9a13-c842c98b1270'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.167709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5603918ebd614f29bafc91c5697cfd10', 'request_id': 'req-c89634f5-4c36-47bb-9a13-c842c98b1270'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.179518 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5603918ebd614f29bafc91c5697cfd10', 'request_id': 'req-c89634f5-4c36-47bb-9a13-c842c98b1270'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.192734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] The token cache is being invalidate because project 5603918ebd614f29bafc91c5697cfd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:10.193182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.194187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.196735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1083/2139] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:09 2022] DELETE /identity/v3/projects/5603918ebd614f29bafc91c5697cfd10 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:10.201153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:10.202644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:10.217219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:10.218179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:10.228855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:10.229458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.231843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.231843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.233461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:10.235864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:10.251162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1057/2140] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:10.252466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:10.254955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:10.267633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fe0b3-38b5-4fd1-9c82-563163683144 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:10.272101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab74a98-975e-4602-b8c7-9f3a9999b426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:10.273119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab74a98-975e-4602-b8c7-9f3a9999b426 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.273482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab74a98-975e-4602-b8c7-9f3a9999b426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.273831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab74a98-975e-4602-b8c7-9f3a9999b426 admin admin] PATH_INFO: `/v3/projects/baff0c96256e4029a829875c2498ad83/users/a867de9112c3449a92861c8a30813814/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.276536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab74a98-975e-4602-b8c7-9f3a9999b426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baff0c96256e4029a829875c2498ad83, user_id=a867de9112c3449a92861c8a30813814, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:10.279834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab74a98-975e-4602-b8c7-9f3a9999b426 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:10.299551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1058/2141] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:10 2022] PUT /identity/v3/projects/baff0c96256e4029a829875c2498ad83/users/a867de9112c3449a92861c8a30813814/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:10.305383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89634f5-4c36-47bb-9a13-c842c98b1270 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:10.306182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff08f44-0a1a-4318-8c7f-5ff9e501981a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.306474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff08f44-0a1a-4318-8c7f-5ff9e501981a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.306887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff08f44-0a1a-4318-8c7f-5ff9e501981a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.337266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1ff08f44-0a1a-4318-8c7f-5ff9e501981a None None] MFA Rules not processed for user `a867de9112c3449a92861c8a30813814`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:10.338468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1ff08f44-0a1a-4318-8c7f-5ff9e501981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:10.381009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:10.382168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.382764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.382895 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] PATH_INFO: `/v3/users/254e3ba112c3460fa32c283628a47ce6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.384269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=254e3ba112c3460fa32c283628a47ce6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:10.386466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:10.436243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1059/2142] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:10.451178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff08f44-0a1a-4318-8c7f-5ff9e501981a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:10.451392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254e3ba112c3460fa32c283628a47ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.451977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254e3ba112c3460fa32c283628a47ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.453061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1ff08f44-0a1a-4318-8c7f-5ff9e501981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:10.476399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254e3ba112c3460fa32c283628a47ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.505641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254e3ba112c3460fa32c283628a47ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.518053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254e3ba112c3460fa32c283628a47ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.530895 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1084/2143] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:10 2022] DELETE /identity/v3/users/254e3ba112c3460fa32c283628a47ce6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:10.541187 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff08f44-0a1a-4318-8c7f-5ff9e501981a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:10.628077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7427fe-b580-4630-ae3e-66924b1568cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:10.628859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7427fe-b580-4630-ae3e-66924b1568cf service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.629229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7427fe-b580-4630-ae3e-66924b1568cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.629511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7427fe-b580-4630-ae3e-66924b1568cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.635288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8b7427fe-b580-4630-ae3e-66924b1568cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:10.678088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:10.716437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7427fe-b580-4630-ae3e-66924b1568cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:10.718204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7427fe-b580-4630-ae3e-66924b1568cf service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:10.728765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1060/2144] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:22:10.793574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:10.795093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:10.848079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:10.873590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6826482-2b04-4184-9842-5aaa01fbaf39 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:10.878154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:10.878957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.879382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.879676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] PATH_INFO: `/v3/users/96baad43289d4a6bb30bb5e6cf396e25` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.880880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96baad43289d4a6bb30bb5e6cf396e25)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:10.882810 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:10.943700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96baad43289d4a6bb30bb5e6cf396e25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.944410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96baad43289d4a6bb30bb5e6cf396e25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.952218 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96baad43289d4a6bb30bb5e6cf396e25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.971456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96baad43289d4a6bb30bb5e6cf396e25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.979409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96baad43289d4a6bb30bb5e6cf396e25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:10.985928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1085/2145] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:10 2022] DELETE /identity/v3/users/96baad43289d4a6bb30bb5e6cf396e25 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:10.990468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43137e35-d49d-4366-9821-8a4f5817e1f7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:10.990760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43137e35-d49d-4366-9821-8a4f5817e1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:10.991055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43137e35-d49d-4366-9821-8a4f5817e1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:10.991629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:10.993550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.009435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-43137e35-d49d-4366-9821-8a4f5817e1f7 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:11.010641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-43137e35-d49d-4366-9821-8a4f5817e1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.089009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abac9d-f751-4b4e-ac70-9497acadc493 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.105914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.106845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.107127 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.107434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] PATH_INFO: `/v3/projects/fae74f3c8aea4a6ca89f867f7665d4f0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.108042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1061/2146] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:11.108981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae74f3c8aea4a6ca89f867f7665d4f0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.109196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5b2ed-5013-407c-8b91-12a4b259a85e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.109484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5b2ed-5013-407c-8b91-12a4b259a85e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.109760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5b2ed-5013-407c-8b91-12a4b259a85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.111794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.139446 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-11f5b2ed-5013-407c-8b91-12a4b259a85e None None] MFA Rules not processed for user `2821263e860a449eb4472ed2c4203e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:11.141720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-11f5b2ed-5013-407c-8b91-12a4b259a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.183983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae74f3c8aea4a6ca89f867f7665d4f0', 'request_id': 'req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:11.191962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae74f3c8aea4a6ca89f867f7665d4f0', 'request_id': 'req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:11.193745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:11.202265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae74f3c8aea4a6ca89f867f7665d4f0', 'request_id': 'req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:11.219275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] The token cache is being invalidate because project fae74f3c8aea4a6ca89f867f7665d4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:11.219699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:11.220486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:11.222456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1086/2147] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:10 2022] DELETE /identity/v3/projects/fae74f3c8aea4a6ca89f867f7665d4f0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:11.224407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.226460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.226952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1062/2148] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:11.232083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87d2ff-f8d0-4c68-93e4-966041103c87 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.232451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87d2ff-f8d0-4c68-93e4-966041103c87 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.232830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87d2ff-f8d0-4c68-93e4-966041103c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.254431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8a87d2ff-f8d0-4c68-93e4-966041103c87 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:11.256159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8a87d2ff-f8d0-4c68-93e4-966041103c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.302163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc7252-2734-4ef5-bfb0-1b60bdda61ab admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.305851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8774fca-45a5-499a-b45b-3fbd292e7d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.306515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774fca-45a5-499a-b45b-3fbd292e7d00 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.306800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774fca-45a5-499a-b45b-3fbd292e7d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.307073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774fca-45a5-499a-b45b-3fbd292e7d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.308997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b8774fca-45a5-499a-b45b-3fbd292e7d00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.329181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1063/2149] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:11.330195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87d2ff-f8d0-4c68-93e4-966041103c87 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.331570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8a87d2ff-f8d0-4c68-93e4-966041103c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.337004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:11.387887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8774fca-45a5-499a-b45b-3fbd292e7d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.389663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8774fca-45a5-499a-b45b-3fbd292e7d00 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.392414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87d2ff-f8d0-4c68-93e4-966041103c87 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.397126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.397780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.398061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.399317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.401521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1087/2150] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:11.401800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.403550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155f72a-7611-4cff-9ee9-2f2da64bb59f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.404011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155f72a-7611-4cff-9ee9-2f2da64bb59f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.404323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155f72a-7611-4cff-9ee9-2f2da64bb59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.423728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9155f72a-7611-4cff-9ee9-2f2da64bb59f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:11.425126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9155f72a-7611-4cff-9ee9-2f2da64bb59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.441805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1088/2151] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:11.443004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155f72a-7611-4cff-9ee9-2f2da64bb59f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.468708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155f72a-7611-4cff-9ee9-2f2da64bb59f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.477423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.478109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.478389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.478657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.479201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.479442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:22:11.479709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.480710 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.481320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.500583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1089/2152] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:22:11.500808 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1064/2153] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:11.504715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.522815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871143d-a520-4262-8ef4-c23c70f4957e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.528702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc14dd-df34-46cb-8ecb-e49d6c73a775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.529893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc14dd-df34-46cb-8ecb-e49d6c73a775 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.530316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc14dd-df34-46cb-8ecb-e49d6c73a775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.530676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc14dd-df34-46cb-8ecb-e49d6c73a775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.531536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc14dd-df34-46cb-8ecb-e49d6c73a775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.534323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc14dd-df34-46cb-8ecb-e49d6c73a775 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.578978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1090/2154] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:11 2022] POST /identity/v3/projects => generated 369 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:11.580240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.592970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0d7b5-9afa-451b-872b-80aa5f60a2ce service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.667362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380d802-667d-4b8e-bbc7-c9363dc310f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.668047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d802-667d-4b8e-bbc7-c9363dc310f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.668381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d802-667d-4b8e-bbc7-c9363dc310f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.668660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d802-667d-4b8e-bbc7-c9363dc310f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.669416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380d802-667d-4b8e-bbc7-c9363dc310f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.671666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380d802-667d-4b8e-bbc7-c9363dc310f7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.694127 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1065/2155] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:11 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:11.698770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc14dd-df34-46cb-8ecb-e49d6c73a775 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.710911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc14dd-df34-46cb-8ecb-e49d6c73a775 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.717581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef67b8-7c11-42e2-9939-56c53ec0b21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.718784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef67b8-7c11-42e2-9939-56c53ec0b21c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.718784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef67b8-7c11-42e2-9939-56c53ec0b21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.719060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef67b8-7c11-42e2-9939-56c53ec0b21c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.719774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef67b8-7c11-42e2-9939-56c53ec0b21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.721435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef67b8-7c11-42e2-9939-56c53ec0b21c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.735886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1091/2156] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:22:11.737093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380d802-667d-4b8e-bbc7-c9363dc310f7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.752055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380d802-667d-4b8e-bbc7-c9363dc310f7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.759461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.759461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.759461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.759461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.760158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.761869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.777305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1066/2157] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:11.778939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef67b8-7c11-42e2-9939-56c53ec0b21c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.791484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef67b8-7c11-42e2-9939-56c53ec0b21c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.801367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2733e1-ce57-4d6c-8263-1038d9245edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.802521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2733e1-ce57-4d6c-8263-1038d9245edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.802891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2733e1-ce57-4d6c-8263-1038d9245edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.803374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2733e1-ce57-4d6c-8263-1038d9245edf admin admin] PATH_INFO: `/v3/projects/f355dd2ce273454196b9a5fbd44f621a/users/705e081253924e349b03bba48480fce6/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.821156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2733e1-ce57-4d6c-8263-1038d9245edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f355dd2ce273454196b9a5fbd44f621a, user_id=705e081253924e349b03bba48480fce6, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.824262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2733e1-ce57-4d6c-8263-1038d9245edf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.829796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.831872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.849587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3f791-ae56-4bc6-916b-368a842cf8ff admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:11.853626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1092/2158] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:11 2022] PUT /identity/v3/projects/f355dd2ce273454196b9a5fbd44f621a/users/705e081253924e349b03bba48480fce6/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:11.865071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5bf7b-a27d-496c-8070-f4d8e873eb0a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.865267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5bf7b-a27d-496c-8070-f4d8e873eb0a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.865578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5bf7b-a27d-496c-8070-f4d8e873eb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.898249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-adb5bf7b-a27d-496c-8070-f4d8e873eb0a None None] MFA Rules not processed for user `705e081253924e349b03bba48480fce6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:11.899408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-adb5bf7b-a27d-496c-8070-f4d8e873eb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:11.908747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:11.911319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:11.940603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:11.941488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:11.941882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:11.947993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] PATH_INFO: `/v3/projects/6f2b5f47c2234f578b4567862cdde573` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:11.949605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2b5f47c2234f578b4567862cdde573)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:11.951723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:11.970399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1093/2159] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:22:11.988220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5bf7b-a27d-496c-8070-f4d8e873eb0a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:11.990377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-adb5bf7b-a27d-496c-8070-f4d8e873eb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:12.007087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2b5f47c2234f578b4567862cdde573', 'request_id': 'req-cef9cb58-d6f1-46ac-8639-fb826e3809f8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.020964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2b5f47c2234f578b4567862cdde573', 'request_id': 'req-cef9cb58-d6f1-46ac-8639-fb826e3809f8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.032025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2b5f47c2234f578b4567862cdde573', 'request_id': 'req-cef9cb58-d6f1-46ac-8639-fb826e3809f8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.046073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] The token cache is being invalidate because project 6f2b5f47c2234f578b4567862cdde573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:12.046499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.047465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.049334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1067/2160] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:11 2022] DELETE /identity/v3/projects/6f2b5f47c2234f578b4567862cdde573 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:12.053744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:12.055244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:12.058849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5bf7b-a27d-496c-8070-f4d8e873eb0a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:12.108725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8cbcc9295074710aeacb35e4e1a4348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:12.109356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:12.109659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:12.110000 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:12.111848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:12.125244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9cb58-d6f1-46ac-8639-fb826e3809f8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:12.130495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:12.131467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:12.131754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:12.132050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] PATH_INFO: `/v3/users/a867de9112c3449a92861c8a30813814` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:12.133265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a867de9112c3449a92861c8a30813814)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:12.135250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:12.172158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:12.173576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:12.182572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1094/2161] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:22:12.183944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a867de9112c3449a92861c8a30813814'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.192847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a867de9112c3449a92861c8a30813814'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.201010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a867de9112c3449a92861c8a30813814'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.201740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a867de9112c3449a92861c8a30813814'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.206440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:12.215752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e2d33ca5-f9db-451f-9d83-94d1d2075c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a867de9112c3449a92861c8a30813814'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.223618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1068/2162] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:12 2022] DELETE /identity/v3/users/a867de9112c3449a92861c8a30813814 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:12.545697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:12.558230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725642f6-0e86-4844-a9de-f1490b6253e6 service glance] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:12.584461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:12.652966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:12.654356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:12.654356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:12.654599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] PATH_INFO: `/v3/projects/baff0c96256e4029a829875c2498ad83` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:12.655892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baff0c96256e4029a829875c2498ad83)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:12.657840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:12.758199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baff0c96256e4029a829875c2498ad83', 'request_id': 'req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.764782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baff0c96256e4029a829875c2498ad83', 'request_id': 'req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.774018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baff0c96256e4029a829875c2498ad83', 'request_id': 'req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.786728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] The token cache is being invalidate because project baff0c96256e4029a829875c2498ad83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:12.787165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.788053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4b7cc0a-4d93-4ae3-a6a8-866e3eb06771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:12.789924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1095/2163] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:12 2022] DELETE /identity/v3/projects/baff0c96256e4029a829875c2498ad83 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:12.795145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225251ea-babc-4778-afc6-409439ebb169 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:12.795721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225251ea-babc-4778-afc6-409439ebb169 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:12.795721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225251ea-babc-4778-afc6-409439ebb169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:12.814895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-225251ea-babc-4778-afc6-409439ebb169 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:12.816372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-225251ea-babc-4778-afc6-409439ebb169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:12.877882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1069/2164] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:22:12.887400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cbd1-ed66-4036-aa84-8e7e23a87b05 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:12.887929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cbd1-ed66-4036-aa84-8e7e23a87b05 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:12.888413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cbd1-ed66-4036-aa84-8e7e23a87b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:12.909001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ceb4cbd1-ed66-4036-aa84-8e7e23a87b05 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:12.910235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ceb4cbd1-ed66-4036-aa84-8e7e23a87b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:12.919488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1096/2165] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:12.924150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225251ea-babc-4778-afc6-409439ebb169 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:12.935255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225251ea-babc-4778-afc6-409439ebb169 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:12.941435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:12.942017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606c098-5270-4e13-ae21-e785295bf1c1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:12.942135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:12.942302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606c098-5270-4e13-ae21-e785295bf1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:12.942410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:12.942571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606c098-5270-4e13-ae21-e785295bf1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:12.942662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:12.943365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:22:12.943680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:12.945306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:12.963435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1070/2166] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:22:12.964555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:12.967588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3606c098-5270-4e13-ae21-e785295bf1c1 None None] MFA Rules not processed for user `2821263e860a449eb4472ed2c4203e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:12.968781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3606c098-5270-4e13-ae21-e785295bf1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:12.975403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179980c6-e50d-4e81-9152-c0ee0d79a1da admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:12.986330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68107-9853-4326-9c60-c6e711598ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:12.986999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68107-9853-4326-9c60-c6e711598ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:12.987318 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68107-9853-4326-9c60-c6e711598ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:12.987580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68107-9853-4326-9c60-c6e711598ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:12.988289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b68107-9853-4326-9c60-c6e711598ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:12.990198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b68107-9853-4326-9c60-c6e711598ce6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.021671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1071/2167] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:12 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:13.025799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68107-9853-4326-9c60-c6e711598ce6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.034465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1097/2168] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:13.106439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68107-9853-4326-9c60-c6e711598ce6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.111862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f34e2-50d6-4b40-8c10-d36be66b76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.112536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f34e2-50d6-4b40-8c10-d36be66b76e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.112821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f34e2-50d6-4b40-8c10-d36be66b76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.113093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f34e2-50d6-4b40-8c10-d36be66b76e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.113787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f34e2-50d6-4b40-8c10-d36be66b76e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.115757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f34e2-50d6-4b40-8c10-d36be66b76e5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.142194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1072/2169] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:13 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:13.143390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606c098-5270-4e13-ae21-e785295bf1c1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.156849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606c098-5270-4e13-ae21-e785295bf1c1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.161824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648c13d-8ad9-410a-8b17-8bd5d3c91f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.162502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c13d-8ad9-410a-8b17-8bd5d3c91f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.162849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c13d-8ad9-410a-8b17-8bd5d3c91f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.163133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c13d-8ad9-410a-8b17-8bd5d3c91f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.163912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648c13d-8ad9-410a-8b17-8bd5d3c91f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.165538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648c13d-8ad9-410a-8b17-8bd5d3c91f7d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.177672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1098/2170] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:13.181459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f34e2-50d6-4b40-8c10-d36be66b76e5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.196043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f34e2-50d6-4b40-8c10-d36be66b76e5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.202898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82be7dd-b405-4e4c-a3f7-a26c6c398301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.203777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be7dd-b405-4e4c-a3f7-a26c6c398301 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.204842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be7dd-b405-4e4c-a3f7-a26c6c398301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.204842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be7dd-b405-4e4c-a3f7-a26c6c398301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.205272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be7dd-b405-4e4c-a3f7-a26c6c398301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.207011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be7dd-b405-4e4c-a3f7-a26c6c398301 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.219252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1073/2171] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:22:13.223143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648c13d-8ad9-410a-8b17-8bd5d3c91f7d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.234448 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648c13d-8ad9-410a-8b17-8bd5d3c91f7d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.239462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9370a1-c005-4288-b839-83c870593b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.240447 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9370a1-c005-4288-b839-83c870593b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.240753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9370a1-c005-4288-b839-83c870593b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.241030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9370a1-c005-4288-b839-83c870593b1f admin admin] PATH_INFO: `/v3/projects/a0f642d1a9034e79b91dd6ace2367821/users/d52a54b6a4e2405bb0f1080f7ddf4874/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.257536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9370a1-c005-4288-b839-83c870593b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f642d1a9034e79b91dd6ace2367821, user_id=d52a54b6a4e2405bb0f1080f7ddf4874, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.260327 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9370a1-c005-4288-b839-83c870593b1f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.283095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1099/2172] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:13 2022] PUT /identity/v3/projects/a0f642d1a9034e79b91dd6ace2367821/users/d52a54b6a4e2405bb0f1080f7ddf4874/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:13.295752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac46fe-506b-4820-b95e-bc336929d1ca None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.296400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac46fe-506b-4820-b95e-bc336929d1ca None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.297036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac46fe-506b-4820-b95e-bc336929d1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.331007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3fac46fe-506b-4820-b95e-bc336929d1ca None None] MFA Rules not processed for user `d52a54b6a4e2405bb0f1080f7ddf4874`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:13.332358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3fac46fe-506b-4820-b95e-bc336929d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:13.350631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:13.407327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1074/2173] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:13.422613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9370a1-c005-4288-b839-83c870593b1f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.438523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:13.494615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9370a1-c005-4288-b839-83c870593b1f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.500468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e855b2-9512-4a29-9fd0-e2c2a215d952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.500913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e855b2-9512-4a29-9fd0-e2c2a215d952 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.501199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e855b2-9512-4a29-9fd0-e2c2a215d952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.501474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e855b2-9512-4a29-9fd0-e2c2a215d952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.502172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e855b2-9512-4a29-9fd0-e2c2a215d952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.504175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e855b2-9512-4a29-9fd0-e2c2a215d952 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.532738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1100/2174] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:13 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:13.536932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac46fe-506b-4820-b95e-bc336929d1ca None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.549272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac46fe-506b-4820-b95e-bc336929d1ca None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.620132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa6f40-f210-41a8-b18e-a34d91ec6413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.621084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa6f40-f210-41a8-b18e-a34d91ec6413 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.621978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa6f40-f210-41a8-b18e-a34d91ec6413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.621978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa6f40-f210-41a8-b18e-a34d91ec6413 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.623025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa6f40-f210-41a8-b18e-a34d91ec6413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.625181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa6f40-f210-41a8-b18e-a34d91ec6413 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.650012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1075/2175] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:13 2022] POST /identity/v3/users => generated 301 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:13.655157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e855b2-9512-4a29-9fd0-e2c2a215d952 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.667502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e855b2-9512-4a29-9fd0-e2c2a215d952 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.673412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a3271-8278-4cae-98bb-8224675017c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.674093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a3271-8278-4cae-98bb-8224675017c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.674379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a3271-8278-4cae-98bb-8224675017c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.674655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a3271-8278-4cae-98bb-8224675017c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.675484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a3271-8278-4cae-98bb-8224675017c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.677134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a3271-8278-4cae-98bb-8224675017c9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.689221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1101/2176] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:22:13.693409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa6f40-f210-41a8-b18e-a34d91ec6413 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.703809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa6f40-f210-41a8-b18e-a34d91ec6413 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.709380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1b153-8656-4633-81bf-903a56863fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.710459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1b153-8656-4633-81bf-903a56863fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.710893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1b153-8656-4633-81bf-903a56863fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.711366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1b153-8656-4633-81bf-903a56863fcf admin admin] PATH_INFO: `/v3/projects/b209da80cedf46b2bea538b969e56140/users/f5422cf99282419296072bbb1d1de139/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.738280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1b153-8656-4633-81bf-903a56863fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b209da80cedf46b2bea538b969e56140, user_id=f5422cf99282419296072bbb1d1de139, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.741067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1b153-8656-4633-81bf-903a56863fcf admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.758924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1076/2177] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:13 2022] PUT /identity/v3/projects/b209da80cedf46b2bea538b969e56140/users/f5422cf99282419296072bbb1d1de139/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:13.763330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a3271-8278-4cae-98bb-8224675017c9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.774174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a3271-8278-4cae-98bb-8224675017c9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.865893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506869c-b6c4-4029-8815-649eeebc24e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.866712 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506869c-b6c4-4029-8815-649eeebc24e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.867090 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506869c-b6c4-4029-8815-649eeebc24e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.867592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506869c-b6c4-4029-8815-649eeebc24e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.868333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506869c-b6c4-4029-8815-649eeebc24e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.870140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506869c-b6c4-4029-8815-649eeebc24e5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.883136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1102/2178] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:13.887188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1b153-8656-4633-81bf-903a56863fcf admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:13.901940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1b153-8656-4633-81bf-903a56863fcf admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:13.907415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:13.908412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.908752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.909087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] PATH_INFO: `/v3/projects/b209da80cedf46b2bea538b969e56140/users/f5422cf99282419296072bbb1d1de139/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.911298 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b209da80cedf46b2bea538b969e56140, user_id=f5422cf99282419296072bbb1d1de139, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:13.914850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:13.928409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1077/2179] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:13 2022] PUT /identity/v3/projects/b209da80cedf46b2bea538b969e56140/users/f5422cf99282419296072bbb1d1de139/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:13.943877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1a34-bdc3-40b7-a8e0-29538b9dedb1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:13.943877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1a34-bdc3-40b7-a8e0-29538b9dedb1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:13.943877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1a34-bdc3-40b7-a8e0-29538b9dedb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:13.976793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b9fc1a34-bdc3-40b7-a8e0-29538b9dedb1 None None] MFA Rules not processed for user `f5422cf99282419296072bbb1d1de139`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:13.978262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b9fc1a34-bdc3-40b7-a8e0-29538b9dedb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:14.064404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1103/2180] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:14.077135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:14.079262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:14.154437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a9694-4cd5-4792-bb73-a08baf2a5d90 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:14.159078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea54b7-5a8c-4086-9ffb-df6eee4577c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:14.159748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea54b7-5a8c-4086-9ffb-df6eee4577c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:14.160032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea54b7-5a8c-4086-9ffb-df6eee4577c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:14.160306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea54b7-5a8c-4086-9ffb-df6eee4577c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:14.179491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea54b7-5a8c-4086-9ffb-df6eee4577c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:14.181147 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea54b7-5a8c-4086-9ffb-df6eee4577c5 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:14.193340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1078/2181] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:14.324614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc1a34-bdc3-40b7-a8e0-29538b9dedb1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:14.335048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:14.392534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc1a34-bdc3-40b7-a8e0-29538b9dedb1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:14.397650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:14.398597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:14.398954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:14.399267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] PATH_INFO: `/v3/users/d52a54b6a4e2405bb0f1080f7ddf4874` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:14.400573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52a54b6a4e2405bb0f1080f7ddf4874)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:14.402659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:14.450658 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52a54b6a4e2405bb0f1080f7ddf4874'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:14.451345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52a54b6a4e2405bb0f1080f7ddf4874'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:14.458248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52a54b6a4e2405bb0f1080f7ddf4874'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:14.469920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52a54b6a4e2405bb0f1080f7ddf4874'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:14.479098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52a54b6a4e2405bb0f1080f7ddf4874'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:14.485752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1104/2182] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:14 2022] DELETE /identity/v3/users/d52a54b6a4e2405bb0f1080f7ddf4874 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:14.490403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41512d96-6418-467a-98a3-0644e1a8ef07 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:14.490777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41512d96-6418-467a-98a3-0644e1a8ef07 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:14.491057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41512d96-6418-467a-98a3-0644e1a8ef07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:14.511582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-41512d96-6418-467a-98a3-0644e1a8ef07 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:14.513472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-41512d96-6418-467a-98a3-0644e1a8ef07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:14.522951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:14.579220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1079/2183] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:22:14.590092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:14.593198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:14.661162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ea368-9525-4f9c-aebb-d3402dd4cbe4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:14.665045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f8061-f887-40b9-a636-d1be295d74d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:14.665693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8061-f887-40b9-a636-d1be295d74d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:14.665979 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8061-f887-40b9-a636-d1be295d74d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:14.666257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8061-f887-40b9-a636-d1be295d74d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:14.676017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f8061-f887-40b9-a636-d1be295d74d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:14.677619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f8061-f887-40b9-a636-d1be295d74d7 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:14.687390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1105/2184] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:15.030723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41512d96-6418-467a-98a3-0644e1a8ef07 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:15.032898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-41512d96-6418-467a-98a3-0644e1a8ef07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:15.051932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41512d96-6418-467a-98a3-0644e1a8ef07 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:15.057592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:15.058811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:15.059397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:15.059916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] PATH_INFO: `/v3/projects/a0f642d1a9034e79b91dd6ace2367821` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:15.061445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f642d1a9034e79b91dd6ace2367821)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:15.063868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:15.135251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f642d1a9034e79b91dd6ace2367821', 'request_id': 'req-7ce85755-2d6b-474d-82d6-1fc2f571681d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.144546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f642d1a9034e79b91dd6ace2367821', 'request_id': 'req-7ce85755-2d6b-474d-82d6-1fc2f571681d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.162202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f642d1a9034e79b91dd6ace2367821', 'request_id': 'req-7ce85755-2d6b-474d-82d6-1fc2f571681d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.176525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] The token cache is being invalidate because project a0f642d1a9034e79b91dd6ace2367821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:15.176816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.177623 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.179681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1080/2185] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:15 2022] DELETE /identity/v3/projects/a0f642d1a9034e79b91dd6ace2367821 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:15.183971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f8061-f887-40b9-a636-d1be295d74d7 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:15.195797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f8061-f887-40b9-a636-d1be295d74d7 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:15.243488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:15.289189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:15.290068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:15.290345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:15.290615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] PATH_INFO: `/v3/users/f5422cf99282419296072bbb1d1de139` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:15.291820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5422cf99282419296072bbb1d1de139)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:15.293743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:15.346380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5422cf99282419296072bbb1d1de139'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.347062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5422cf99282419296072bbb1d1de139'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.354331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5422cf99282419296072bbb1d1de139'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.368693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5422cf99282419296072bbb1d1de139'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.377948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a561bf0f-2a90-4e8c-a5a5-092133ea5dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5422cf99282419296072bbb1d1de139'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.384711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1106/2186] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:15 2022] DELETE /identity/v3/users/f5422cf99282419296072bbb1d1de139 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:15.688792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:15.690704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:15.707244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85755-2d6b-474d-82d6-1fc2f571681d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:15.773112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:15.774149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:15.774466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:15.774804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] PATH_INFO: `/v3/projects/b209da80cedf46b2bea538b969e56140` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:15.776077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b209da80cedf46b2bea538b969e56140)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:15.778017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:15.821443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:15.831803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b209da80cedf46b2bea538b969e56140', 'request_id': 'req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.842139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b209da80cedf46b2bea538b969e56140', 'request_id': 'req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.867167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b209da80cedf46b2bea538b969e56140', 'request_id': 'req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.895772 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] The token cache is being invalidate because project b209da80cedf46b2bea538b969e56140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:15.896067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.896771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-927fc81d-3073-4fd1-a2c1-58d7a7b32cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:15.898635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1081/2187] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:15 2022] DELETE /identity/v3/projects/b209da80cedf46b2bea538b969e56140 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:15.903590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9ef53-1ff8-4387-b964-90d32d31b97e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:15.903872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9ef53-1ff8-4387-b964-90d32d31b97e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:15.904415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9ef53-1ff8-4387-b964-90d32d31b97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:15.922370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d5f9ef53-1ff8-4387-b964-90d32d31b97e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:15.923644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d5f9ef53-1ff8-4387-b964-90d32d31b97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:16.011404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1107/2188] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:16.019933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373c3d3-1ec4-4ab9-bc31-15319173452a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.020224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373c3d3-1ec4-4ab9-bc31-15319173452a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.020508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373c3d3-1ec4-4ab9-bc31-15319173452a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.041442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b373c3d3-1ec4-4ab9-bc31-15319173452a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:16.042682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b373c3d3-1ec4-4ab9-bc31-15319173452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:16.055148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1082/2189] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:22:16.063236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9ef53-1ff8-4387-b964-90d32d31b97e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.075498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9ef53-1ff8-4387-b964-90d32d31b97e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.080873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.081622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.082008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.082382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.083194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:22:16.083592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.085287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.101407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1108/2190] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:22:16.102356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373c3d3-1ec4-4ab9-bc31-15319173452a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.112839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373c3d3-1ec4-4ab9-bc31-15319173452a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.118397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce62a0-9074-45eb-8c57-ef35193a93d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.119084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce62a0-9074-45eb-8c57-ef35193a93d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.119406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce62a0-9074-45eb-8c57-ef35193a93d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.119684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce62a0-9074-45eb-8c57-ef35193a93d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.120407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce62a0-9074-45eb-8c57-ef35193a93d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.122946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce62a0-9074-45eb-8c57-ef35193a93d8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.157056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1083/2191] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:16 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:22:16.161173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.171750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca28a3e-f11a-4e64-9ee4-351cc68b390e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.209483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:16.244174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8084d5-9d55-427f-a054-a35acb7f83f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.244651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8084d5-9d55-427f-a054-a35acb7f83f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.244937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8084d5-9d55-427f-a054-a35acb7f83f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.245219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8084d5-9d55-427f-a054-a35acb7f83f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.245929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8084d5-9d55-427f-a054-a35acb7f83f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.257956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8084d5-9d55-427f-a054-a35acb7f83f0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.297358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1109/2192] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:16 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:16.301391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce62a0-9074-45eb-8c57-ef35193a93d8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.315100 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce62a0-9074-45eb-8c57-ef35193a93d8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.321278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e671d61-c4e8-41db-8398-1ea673e0e926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.321972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e671d61-c4e8-41db-8398-1ea673e0e926 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.322255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e671d61-c4e8-41db-8398-1ea673e0e926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.323611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e671d61-c4e8-41db-8398-1ea673e0e926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.324811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e671d61-c4e8-41db-8398-1ea673e0e926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.326574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e671d61-c4e8-41db-8398-1ea673e0e926 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.344520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8084d5-9d55-427f-a054-a35acb7f83f0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.351157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1084/2193] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:16.361693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8084d5-9d55-427f-a054-a35acb7f83f0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.374929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b6050-aa7c-4329-9fdf-583ebc72ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.376996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b6050-aa7c-4329-9fdf-583ebc72ce20 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.377578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b6050-aa7c-4329-9fdf-583ebc72ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.377989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b6050-aa7c-4329-9fdf-583ebc72ce20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.378878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b6050-aa7c-4329-9fdf-583ebc72ce20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.380949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b6050-aa7c-4329-9fdf-583ebc72ce20 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.395358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1110/2194] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:22:16.400361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e671d61-c4e8-41db-8398-1ea673e0e926 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.412126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e671d61-c4e8-41db-8398-1ea673e0e926 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.417331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a087a-4477-4e0e-870c-c6941e30169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.418262 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a087a-4477-4e0e-870c-c6941e30169c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.418549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a087a-4477-4e0e-870c-c6941e30169c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.418831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a087a-4477-4e0e-870c-c6941e30169c admin admin] PATH_INFO: `/v3/projects/6d98433e08774bad96203e8028d6376c/users/84fb3ae64fe94332a123b94390ec67d5/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.436908 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a087a-4477-4e0e-870c-c6941e30169c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d98433e08774bad96203e8028d6376c, user_id=84fb3ae64fe94332a123b94390ec67d5, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.443575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a087a-4477-4e0e-870c-c6941e30169c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.462518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1085/2195] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:16 2022] PUT /identity/v3/projects/6d98433e08774bad96203e8028d6376c/users/84fb3ae64fe94332a123b94390ec67d5/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:16.474438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae44e10-bd18-4139-95f1-c4d21dff7b66 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.474732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae44e10-bd18-4139-95f1-c4d21dff7b66 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.475040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae44e10-bd18-4139-95f1-c4d21dff7b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.514580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cae44e10-bd18-4139-95f1-c4d21dff7b66 None None] MFA Rules not processed for user `84fb3ae64fe94332a123b94390ec67d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:16.515903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cae44e10-bd18-4139-95f1-c4d21dff7b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:16.591190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1111/2196] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:16.596847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a087a-4477-4e0e-870c-c6941e30169c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.675944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a087a-4477-4e0e-870c-c6941e30169c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.681521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47222363-e0ac-4b37-86bd-373844ef2698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.682142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47222363-e0ac-4b37-86bd-373844ef2698 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.682585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47222363-e0ac-4b37-86bd-373844ef2698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.682867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47222363-e0ac-4b37-86bd-373844ef2698 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.683609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47222363-e0ac-4b37-86bd-373844ef2698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.685590 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47222363-e0ac-4b37-86bd-373844ef2698 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.722016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1086/2197] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:16 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:16.724942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae44e10-bd18-4139-95f1-c4d21dff7b66 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.736059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae44e10-bd18-4139-95f1-c4d21dff7b66 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.796800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ead2-93f0-459b-9f29-58da9826c168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.797586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ead2-93f0-459b-9f29-58da9826c168 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.803376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ead2-93f0-459b-9f29-58da9826c168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.803376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ead2-93f0-459b-9f29-58da9826c168 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.803376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121ead2-93f0-459b-9f29-58da9826c168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.803376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121ead2-93f0-459b-9f29-58da9826c168 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.826826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1112/2198] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:16 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:16.831028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47222363-e0ac-4b37-86bd-373844ef2698 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.842451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47222363-e0ac-4b37-86bd-373844ef2698 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.847690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4d25e-86a5-4d9e-849d-c16eaf2c974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.848365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d25e-86a5-4d9e-849d-c16eaf2c974c admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.848650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d25e-86a5-4d9e-849d-c16eaf2c974c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.848921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d25e-86a5-4d9e-849d-c16eaf2c974c admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.849660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4d25e-86a5-4d9e-849d-c16eaf2c974c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.851295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4d25e-86a5-4d9e-849d-c16eaf2c974c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.862078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1087/2199] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:16.866226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ead2-93f0-459b-9f29-58da9826c168 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.877156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ead2-93f0-459b-9f29-58da9826c168 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.882565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8120b6f-9bae-4e3e-b19a-6e39fe17f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:16.883560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8120b6f-9bae-4e3e-b19a-6e39fe17f9a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:16.883956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8120b6f-9bae-4e3e-b19a-6e39fe17f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:16.884255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8120b6f-9bae-4e3e-b19a-6e39fe17f9a5 admin admin] PATH_INFO: `/v3/projects/bb271a9952184e1aacbb27b9ef826fbe/users/ad075939bbcb4a14823dbe199b040256/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:16.903856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8120b6f-9bae-4e3e-b19a-6e39fe17f9a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb271a9952184e1aacbb27b9ef826fbe, user_id=ad075939bbcb4a14823dbe199b040256, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:16.906760 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8120b6f-9bae-4e3e-b19a-6e39fe17f9a5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:16.923559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1113/2200] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:16 2022] PUT /identity/v3/projects/bb271a9952184e1aacbb27b9ef826fbe/users/ad075939bbcb4a14823dbe199b040256/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:16.927973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4d25e-86a5-4d9e-849d-c16eaf2c974c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:16.938681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4d25e-86a5-4d9e-849d-c16eaf2c974c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:16.975053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:17.021746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b37b20-9236-49a6-aef1-586a74d2f73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:17.022444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b37b20-9236-49a6-aef1-586a74d2f73e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:17.022729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b37b20-9236-49a6-aef1-586a74d2f73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:17.023004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b37b20-9236-49a6-aef1-586a74d2f73e admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:17.023827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b37b20-9236-49a6-aef1-586a74d2f73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:17.025437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b37b20-9236-49a6-aef1-586a74d2f73e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:17.039101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1088/2201] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:17.040429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8120b6f-9bae-4e3e-b19a-6e39fe17f9a5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:17.051178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8120b6f-9bae-4e3e-b19a-6e39fe17f9a5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:17.062397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:17.063397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:17.064397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:17.064397 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] PATH_INFO: `/v3/projects/bb271a9952184e1aacbb27b9ef826fbe/users/ad075939bbcb4a14823dbe199b040256/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:17.066275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb271a9952184e1aacbb27b9ef826fbe, user_id=ad075939bbcb4a14823dbe199b040256, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:17.069304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:17.077060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:17.083502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1114/2202] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:17 2022] PUT /identity/v3/projects/bb271a9952184e1aacbb27b9ef826fbe/users/ad075939bbcb4a14823dbe199b040256/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:17.094363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552eeba1-bffb-4e88-9552-759c11306f6a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:17.094928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552eeba1-bffb-4e88-9552-759c11306f6a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:17.094928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552eeba1-bffb-4e88-9552-759c11306f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:17.127837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-552eeba1-bffb-4e88-9552-759c11306f6a None None] MFA Rules not processed for user `ad075939bbcb4a14823dbe199b040256`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:17.129290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-552eeba1-bffb-4e88-9552-759c11306f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:17.220490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1089/2203] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:17.234466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:17.235893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:17.319114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f5a9b-60f7-41f5-8421-7147dacce558 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:17.323047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de2661-8081-4477-b5e6-eba28016b8ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:17.323743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de2661-8081-4477-b5e6-eba28016b8ec service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:17.324147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de2661-8081-4477-b5e6-eba28016b8ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:17.324430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de2661-8081-4477-b5e6-eba28016b8ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:17.335140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de2661-8081-4477-b5e6-eba28016b8ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:17.336619 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de2661-8081-4477-b5e6-eba28016b8ec service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:17.346408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1115/2204] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:17.792321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552eeba1-bffb-4e88-9552-759c11306f6a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:17.865456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552eeba1-bffb-4e88-9552-759c11306f6a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:17.870058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:17.870946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:17.871247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:17.871529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] PATH_INFO: `/v3/users/84fb3ae64fe94332a123b94390ec67d5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:17.873113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84fb3ae64fe94332a123b94390ec67d5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:17.875060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:17.932834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84fb3ae64fe94332a123b94390ec67d5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:17.942050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84fb3ae64fe94332a123b94390ec67d5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:17.947739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:17.949801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84fb3ae64fe94332a123b94390ec67d5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:17.951071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84fb3ae64fe94332a123b94390ec67d5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:17.964281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84fb3ae64fe94332a123b94390ec67d5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:17.970762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1090/2205] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:17 2022] DELETE /identity/v3/users/84fb3ae64fe94332a123b94390ec67d5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:17.975439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb50c2-dc5b-4e78-87a9-f79e029b7756 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:17.975700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb50c2-dc5b-4e78-87a9-f79e029b7756 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:17.975982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb50c2-dc5b-4e78-87a9-f79e029b7756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:17.995068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b2eb50c2-dc5b-4e78-87a9-f79e029b7756 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:17.996319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2eb50c2-dc5b-4e78-87a9-f79e029b7756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:18.064258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1116/2206] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:18.074026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:18.075665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:18.135253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27666534-e066-4e62-97e0-1e91b1432703 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:18.139153 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdf83-9be0-4692-8ba1-7f9464a1d2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:18.139829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdf83-9be0-4692-8ba1-7f9464a1d2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:18.140104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdf83-9be0-4692-8ba1-7f9464a1d2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:18.140374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdf83-9be0-4692-8ba1-7f9464a1d2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:18.149696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acdf83-9be0-4692-8ba1-7f9464a1d2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:18.151129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acdf83-9be0-4692-8ba1-7f9464a1d2f5 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:18.160647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1091/2207] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:18.532161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb50c2-dc5b-4e78-87a9-f79e029b7756 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:18.534107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2eb50c2-dc5b-4e78-87a9-f79e029b7756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:18.549525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb50c2-dc5b-4e78-87a9-f79e029b7756 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:18.554776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:18.555766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:18.556124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:18.556398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] PATH_INFO: `/v3/projects/6d98433e08774bad96203e8028d6376c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:18.557906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d98433e08774bad96203e8028d6376c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:18.560111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:18.620251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d98433e08774bad96203e8028d6376c', 'request_id': 'req-272339f4-4c1a-4600-b111-ea68adde8f95'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.627131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d98433e08774bad96203e8028d6376c', 'request_id': 'req-272339f4-4c1a-4600-b111-ea68adde8f95'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.636432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d98433e08774bad96203e8028d6376c', 'request_id': 'req-272339f4-4c1a-4600-b111-ea68adde8f95'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.649963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] The token cache is being invalidate because project 6d98433e08774bad96203e8028d6376c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:18.650153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.650859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.652474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1117/2208] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:18 2022] DELETE /identity/v3/projects/6d98433e08774bad96203e8028d6376c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:18.657351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdf83-9be0-4692-8ba1-7f9464a1d2f5 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:18.667179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdf83-9be0-4692-8ba1-7f9464a1d2f5 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:18.731709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:18.732790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:18.733072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:18.733349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] PATH_INFO: `/v3/users/ad075939bbcb4a14823dbe199b040256` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:18.734738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad075939bbcb4a14823dbe199b040256)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:18.736706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:18.787702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad075939bbcb4a14823dbe199b040256'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.800259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad075939bbcb4a14823dbe199b040256'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.809501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad075939bbcb4a14823dbe199b040256'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.810786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad075939bbcb4a14823dbe199b040256'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.824526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-87ac7cea-69cf-4ff0-a851-6e313d36de14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad075939bbcb4a14823dbe199b040256'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:18.827999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:18.833135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1092/2209] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:22:18 2022] DELETE /identity/v3/users/ad075939bbcb4a14823dbe199b040256 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:22:19.143898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:19.145453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:19.159476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272339f4-4c1a-4600-b111-ea68adde8f95 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:19.183344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:19.225572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:19.226444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.226722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.226995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] PATH_INFO: `/v3/projects/bb271a9952184e1aacbb27b9ef826fbe` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.228232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb271a9952184e1aacbb27b9ef826fbe)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:19.230314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:19.281164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb271a9952184e1aacbb27b9ef826fbe', 'request_id': 'req-2e7ea077-17dd-44b6-84b4-debfd042aaa8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:19.287173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb271a9952184e1aacbb27b9ef826fbe', 'request_id': 'req-2e7ea077-17dd-44b6-84b4-debfd042aaa8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:19.296146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb271a9952184e1aacbb27b9ef826fbe', 'request_id': 'req-2e7ea077-17dd-44b6-84b4-debfd042aaa8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:19.307360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] The token cache is being invalidate because project bb271a9952184e1aacbb27b9ef826fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:22:19.307647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:19.308269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2e7ea077-17dd-44b6-84b4-debfd042aaa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:22:19.310466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1118/2210] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:22:19 2022] DELETE /identity/v3/projects/bb271a9952184e1aacbb27b9ef826fbe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:19.315517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043d05-2f7c-4637-9a6c-f47c22548a95 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.316002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043d05-2f7c-4637-9a6c-f47c22548a95 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.316096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043d05-2f7c-4637-9a6c-f47c22548a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.335156 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b1043d05-2f7c-4637-9a6c-f47c22548a95 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:19.336398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b1043d05-2f7c-4637-9a6c-f47c22548a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:19.406781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1093/2211] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:19.415121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19c20-48f0-4418-b404-bbe339629550 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.415563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19c20-48f0-4418-b404-bbe339629550 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.415847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19c20-48f0-4418-b404-bbe339629550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.436561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6ec19c20-48f0-4418-b404-bbe339629550 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:19.437779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6ec19c20-48f0-4418-b404-bbe339629550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:19.446932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1119/2212] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:19.452706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043d05-2f7c-4637-9a6c-f47c22548a95 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:19.465803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043d05-2f7c-4637-9a6c-f47c22548a95 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:19.469173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:19.469859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.470141 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.470421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.471185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:22:19.471541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:19.473177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:19.486789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1094/2213] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:22:19.491512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec19c20-48f0-4418-b404-bbe339629550 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:19.501978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec19c20-48f0-4418-b404-bbe339629550 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:19.507300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc913bb-68c6-4d50-855e-70d9af4e5a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:19.508118 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc913bb-68c6-4d50-855e-70d9af4e5a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.508400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc913bb-68c6-4d50-855e-70d9af4e5a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.508672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc913bb-68c6-4d50-855e-70d9af4e5a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.509371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc913bb-68c6-4d50-855e-70d9af4e5a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:19.511847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc913bb-68c6-4d50-855e-70d9af4e5a17 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:19.539302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1120/2214] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:22:19 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:19.543311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:19.553642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0223-8475-40d3-b24e-23c9e360ccf4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:19.615152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a717576-da30-42b0-8328-0ff313aa5b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:19.615860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717576-da30-42b0-8328-0ff313aa5b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.616139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717576-da30-42b0-8328-0ff313aa5b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.616410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717576-da30-42b0-8328-0ff313aa5b89 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.617107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a717576-da30-42b0-8328-0ff313aa5b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:19.619058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a717576-da30-42b0-8328-0ff313aa5b89 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:19.643287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1095/2215] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:22:19 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:22:19.645457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc913bb-68c6-4d50-855e-70d9af4e5a17 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:19.664078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc913bb-68c6-4d50-855e-70d9af4e5a17 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:19.669188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f6e75-5417-447d-a519-ab70c8708e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:19.669853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f6e75-5417-447d-a519-ab70c8708e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.670135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f6e75-5417-447d-a519-ab70c8708e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.670410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f6e75-5417-447d-a519-ab70c8708e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.671141 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f6e75-5417-447d-a519-ab70c8708e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:19.672780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f6e75-5417-447d-a519-ab70c8708e34 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:19.685375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1121/2216] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:19.689541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a717576-da30-42b0-8328-0ff313aa5b89 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:19.700266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a717576-da30-42b0-8328-0ff313aa5b89 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:19.705075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a295f-1971-4dc9-819b-dc8849bf016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:19.705761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a295f-1971-4dc9-819b-dc8849bf016a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.706042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a295f-1971-4dc9-819b-dc8849bf016a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.706317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a295f-1971-4dc9-819b-dc8849bf016a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.707078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a295f-1971-4dc9-819b-dc8849bf016a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:19.708776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a295f-1971-4dc9-819b-dc8849bf016a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:19.718919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1096/2217] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:22:19.722959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f6e75-5417-447d-a519-ab70c8708e34 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:19.733597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f6e75-5417-447d-a519-ab70c8708e34 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:19.738937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:19.739873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.740210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.740482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] PATH_INFO: `/v3/projects/0c6c7256c24245f192960e6df9211c29/users/959b1de76934495ca7f3b821ff27827f/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.757432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6c7256c24245f192960e6df9211c29, user_id=959b1de76934495ca7f3b821ff27827f, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:19.760171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:19.777972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1122/2218] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:22:19 2022] PUT /identity/v3/projects/0c6c7256c24245f192960e6df9211c29/users/959b1de76934495ca7f3b821ff27827f/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:22:19.789312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de657f-6480-4b34-9754-ad7b464efe87 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:19.789671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de657f-6480-4b34-9754-ad7b464efe87 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:19.790087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de657f-6480-4b34-9754-ad7b464efe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:19.818967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e0de657f-6480-4b34-9754-ad7b464efe87 None None] MFA Rules not processed for user `959b1de76934495ca7f3b821ff27827f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:19.820204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e0de657f-6480-4b34-9754-ad7b464efe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:19.877621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:19.890403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1097/2219] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:21.250595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:22:21.252696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:21.319303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990ca00-cdee-416b-a9c6-c01c41e3d880 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:22:21.323461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8cbcc9295074710aeacb35e4e1a4348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:22:21.324280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:21.324565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:21.324809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:21.330685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:21.334961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:22:21.336891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:22:21.354826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1123/2220] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:22:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:22:29.890598 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:31.352346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:22:38.873708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1429-53e8-4090-863d-41803a7191be None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:22:38.874346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1429-53e8-4090-863d-41803a7191be None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:22:38.874346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1429-53e8-4090-863d-41803a7191be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:22:38.893659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-85ef1429-53e8-4090-863d-41803a7191be None None] MFA Rules not processed for user `2821263e860a449eb4472ed2c4203e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:22:38.894999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-85ef1429-53e8-4090-863d-41803a7191be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:22:38.955152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1098/2221] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:22:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:22:48.948106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=25 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:24.945365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:24.947122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:25.095355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd267d-815b-48a3-b02b-d3ccafed6464 service glance] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:25.101204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc54e85-a242-48d0-965e-62c584f39a24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:25.101842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54e85-a242-48d0-965e-62c584f39a24 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:25.102124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54e85-a242-48d0-965e-62c584f39a24 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:25.102479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54e85-a242-48d0-965e-62c584f39a24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:25.104399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8bc54e85-a242-48d0-965e-62c584f39a24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:25.115890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc54e85-a242-48d0-965e-62c584f39a24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:25.117479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc54e85-a242-48d0-965e-62c584f39a24 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:25.128004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1124/2222] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:23:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:23:29.670434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dec4dd-7fad-4fd9-8722-5b88dad6e657 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:29.671020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dec4dd-7fad-4fd9-8722-5b88dad6e657 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:29.671020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dec4dd-7fad-4fd9-8722-5b88dad6e657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:29.690490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-54dec4dd-7fad-4fd9-8722-5b88dad6e657 None None] MFA Rules not processed for user `a15a0abc2f25410b8476464d7cd32802`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:29.691766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-54dec4dd-7fad-4fd9-8722-5b88dad6e657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:29.751081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1099/2223] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:33.027554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347af3d-7658-4e39-add4-569c697c5cd0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:33.028439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347af3d-7658-4e39-add4-569c697c5cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:33.028983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347af3d-7658-4e39-add4-569c697c5cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:33.059562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7347af3d-7658-4e39-add4-569c697c5cd0 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:33.061006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7347af3d-7658-4e39-add4-569c697c5cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:33.159602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1125/2224] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:23:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:23:33.221743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dec4dd-7fad-4fd9-8722-5b88dad6e657 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:33.223980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-54dec4dd-7fad-4fd9-8722-5b88dad6e657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:33.318118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dec4dd-7fad-4fd9-8722-5b88dad6e657 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:33.325289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94342af1-1af0-4646-89f6-a25e309b9c68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:33.326077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94342af1-1af0-4646-89f6-a25e309b9c68 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:33.326322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94342af1-1af0-4646-89f6-a25e309b9c68 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:33.326596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94342af1-1af0-4646-89f6-a25e309b9c68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:33.337552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94342af1-1af0-4646-89f6-a25e309b9c68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:33.340199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94342af1-1af0-4646-89f6-a25e309b9c68 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:33.362240 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1100/2225] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:23:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:23:43.151806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:43.352396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=57 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:45.608714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347af3d-7658-4e39-add4-569c697c5cd0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:45.611135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7347af3d-7658-4e39-add4-569c697c5cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:45.734077 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347af3d-7658-4e39-add4-569c697c5cd0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:45.740011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:45.741215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:45.741636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:45.741919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] PATH_INFO: `/v3/users/705e081253924e349b03bba48480fce6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:45.743176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] RBAC: Authorizing `identity:delete_user(user_id=705e081253924e349b03bba48480fce6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:45.745684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:45.808999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '705e081253924e349b03bba48480fce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:45.809667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '705e081253924e349b03bba48480fce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:45.819323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '705e081253924e349b03bba48480fce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:45.834097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '705e081253924e349b03bba48480fce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:45.843978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '705e081253924e349b03bba48480fce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:45.855250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b45e8-1da8-476e-81b0-9229a5ccac3d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:45.855854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b45e8-1da8-476e-81b0-9229a5ccac3d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:45.856196 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b45e8-1da8-476e-81b0-9229a5ccac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:45.857753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1126/2226] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:23:45 2022] DELETE /identity/v3/users/705e081253924e349b03bba48480fce6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:23:45.883629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-be4b45e8-1da8-476e-81b0-9229a5ccac3d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:45.885041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-be4b45e8-1da8-476e-81b0-9229a5ccac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:45.950634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1101/2227] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:45.960951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:45.962469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:46.043272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:46.092671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27622bb1-2483-496a-8765-0e847911234d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:46.096678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092eded-e28a-420f-87b8-a8872654cbb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:46.097331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092eded-e28a-420f-87b8-a8872654cbb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:46.097609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092eded-e28a-420f-87b8-a8872654cbb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:46.097879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092eded-e28a-420f-87b8-a8872654cbb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:46.108142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092eded-e28a-420f-87b8-a8872654cbb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:46.111093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092eded-e28a-420f-87b8-a8872654cbb3 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:46.122868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1127/2228] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:23:46.559693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b45e8-1da8-476e-81b0-9229a5ccac3d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:46.562212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-be4b45e8-1da8-476e-81b0-9229a5ccac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:46.577522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b45e8-1da8-476e-81b0-9229a5ccac3d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:46.585215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:46.586464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:46.587046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:46.587582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] PATH_INFO: `/v3/projects/f355dd2ce273454196b9a5fbd44f621a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:46.589706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f355dd2ce273454196b9a5fbd44f621a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:46.591980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:46.680465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f355dd2ce273454196b9a5fbd44f621a', 'request_id': 'req-be39405e-77b8-4710-a52a-6f9e072773ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:46.687352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f355dd2ce273454196b9a5fbd44f621a', 'request_id': 'req-be39405e-77b8-4710-a52a-6f9e072773ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:46.700077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f355dd2ce273454196b9a5fbd44f621a', 'request_id': 'req-be39405e-77b8-4710-a52a-6f9e072773ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:46.730278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] The token cache is being invalidate because project f355dd2ce273454196b9a5fbd44f621a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:23:46.730568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:46.734402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be39405e-77b8-4710-a52a-6f9e072773ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:46.736171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1102/2229] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:23:46 2022] DELETE /identity/v3/projects/f355dd2ce273454196b9a5fbd44f621a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:23:46.741169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa16ea-6b47-446e-a85a-0585b3766800 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:46.741700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa16ea-6b47-446e-a85a-0585b3766800 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:46.741848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa16ea-6b47-446e-a85a-0585b3766800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:46.830553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1dfa16ea-6b47-446e-a85a-0585b3766800 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:46.831869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1dfa16ea-6b47-446e-a85a-0585b3766800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:46.958309 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1128/2230] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:46.965900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680dddfd-0dfc-49d8-8d74-085fc4cad07a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:46.966381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680dddfd-0dfc-49d8-8d74-085fc4cad07a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:46.966486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680dddfd-0dfc-49d8-8d74-085fc4cad07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:46.991142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-680dddfd-0dfc-49d8-8d74-085fc4cad07a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:46.994860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-680dddfd-0dfc-49d8-8d74-085fc4cad07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:47.020927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1103/2231] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:47.022526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa16ea-6b47-446e-a85a-0585b3766800 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:47.056597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa16ea-6b47-446e-a85a-0585b3766800 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:47.064065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:47.064778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:47.065120 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:47.065402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:47.066153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:23:47.066464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:47.068212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:47.107696 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1129/2232] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:23:47.110406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680dddfd-0dfc-49d8-8d74-085fc4cad07a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:47.132333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680dddfd-0dfc-49d8-8d74-085fc4cad07a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:47.153701 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944e672-9b49-44e6-a1ac-230aa8dd3871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:47.169773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944e672-9b49-44e6-a1ac-230aa8dd3871 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:47.170541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944e672-9b49-44e6-a1ac-230aa8dd3871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:47.174063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944e672-9b49-44e6-a1ac-230aa8dd3871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:47.175946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944e672-9b49-44e6-a1ac-230aa8dd3871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:47.178169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944e672-9b49-44e6-a1ac-230aa8dd3871 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:47.252329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1104/2233] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:23:47 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:23:47.252956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:47.269220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a963c-6bcc-42f3-821f-4d6f365df075 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:47.382990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84eb85-73ec-466d-b304-40d8b1c9895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:47.383796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84eb85-73ec-466d-b304-40d8b1c9895b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:47.384147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84eb85-73ec-466d-b304-40d8b1c9895b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:47.384491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84eb85-73ec-466d-b304-40d8b1c9895b admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:47.385368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84eb85-73ec-466d-b304-40d8b1c9895b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:47.389488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84eb85-73ec-466d-b304-40d8b1c9895b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:47.460796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944e672-9b49-44e6-a1ac-230aa8dd3871 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:47.462550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1130/2234] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:23:47 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:23:47.473869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944e672-9b49-44e6-a1ac-230aa8dd3871 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:47.495800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170ef96-c2f9-42a7-9e9c-6ad1c9f20c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:47.512648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170ef96-c2f9-42a7-9e9c-6ad1c9f20c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:47.512895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170ef96-c2f9-42a7-9e9c-6ad1c9f20c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:47.513510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170ef96-c2f9-42a7-9e9c-6ad1c9f20c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:47.514536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170ef96-c2f9-42a7-9e9c-6ad1c9f20c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:47.516374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170ef96-c2f9-42a7-9e9c-6ad1c9f20c16 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:47.549480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1105/2235] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:23:47.553542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84eb85-73ec-466d-b304-40d8b1c9895b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:47.571436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84eb85-73ec-466d-b304-40d8b1c9895b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:47.572350 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0cd1c-0795-43bb-a72f-552b97e7cdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:47.573354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cd1c-0795-43bb-a72f-552b97e7cdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:47.573691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cd1c-0795-43bb-a72f-552b97e7cdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:47.573970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cd1c-0795-43bb-a72f-552b97e7cdd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:47.580412 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0cd1c-0795-43bb-a72f-552b97e7cdd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:47.582122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0cd1c-0795-43bb-a72f-552b97e7cdd5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:47.601047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1131/2236] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:23:47.601503 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170ef96-c2f9-42a7-9e9c-6ad1c9f20c16 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:47.616727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170ef96-c2f9-42a7-9e9c-6ad1c9f20c16 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:47.624526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:47.625776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:47.626430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:47.627020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] PATH_INFO: `/v3/projects/bf86466e66274193b5c2b0fdf69884b5/users/578195adbe574c67a6de17a17513f545/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:47.648343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf86466e66274193b5c2b0fdf69884b5, user_id=578195adbe574c67a6de17a17513f545, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:47.655950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:47.693946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723b141-5b31-4de9-96c7-ecd00c9e6dc1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:47.694264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723b141-5b31-4de9-96c7-ecd00c9e6dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:47.694548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723b141-5b31-4de9-96c7-ecd00c9e6dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:47.700328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1106/2237] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:23:47 2022] PUT /identity/v3/projects/bf86466e66274193b5c2b0fdf69884b5/users/578195adbe574c67a6de17a17513f545/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:23:47.737417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4723b141-5b31-4de9-96c7-ecd00c9e6dc1 None None] MFA Rules not processed for user `578195adbe574c67a6de17a17513f545`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:47.738864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4723b141-5b31-4de9-96c7-ecd00c9e6dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:47.743323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:47.830281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1132/2238] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:47.842171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:47.844178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:47.868215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:47.991745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efca9ab-d1fe-4c8d-a9ca-865169992cb4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:48.001785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:48.002904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:48.003867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:48.004429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:48.017690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:48.019829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:48.045220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1107/2239] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:23:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:23:49.626100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf04a-e470-4654-bdbd-62a9a14558c0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:49.626746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf04a-e470-4654-bdbd-62a9a14558c0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:49.626856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf04a-e470-4654-bdbd-62a9a14558c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:49.648214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-18cdf04a-e470-4654-bdbd-62a9a14558c0 None None] MFA Rules not processed for user `578195adbe574c67a6de17a17513f545`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:49.650133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-18cdf04a-e470-4654-bdbd-62a9a14558c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:49.666192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1133/2240] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:52.390726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:52.392405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:52.495815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec50f6a-e812-492b-8595-8a93fa37af93 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:52.501507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:52.502627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:52.503073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:52.503539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] PATH_INFO: `/v3/users/959b1de76934495ca7f3b821ff27827f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:52.505439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=959b1de76934495ca7f3b821ff27827f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:52.507818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:52.635456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '959b1de76934495ca7f3b821ff27827f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:52.646290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '959b1de76934495ca7f3b821ff27827f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:52.673726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '959b1de76934495ca7f3b821ff27827f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:52.675176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '959b1de76934495ca7f3b821ff27827f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:52.689746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '959b1de76934495ca7f3b821ff27827f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:52.698725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1108/2241] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:23:52 2022] DELETE /identity/v3/users/959b1de76934495ca7f3b821ff27827f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:23:52.704637 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee95d7-4bed-4e42-b0a4-aa16e515e37d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:52.705159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee95d7-4bed-4e42-b0a4-aa16e515e37d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:52.705362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee95d7-4bed-4e42-b0a4-aa16e515e37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:52.726209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-21ee95d7-4bed-4e42-b0a4-aa16e515e37d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:52.727731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-21ee95d7-4bed-4e42-b0a4-aa16e515e37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:52.808892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1134/2242] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:52.821613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:52.823295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:52.849502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:52.898386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0812d16-1518-4b2c-bb85-2d2b80b22c3b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:52.910294 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec31df3-93d0-4a5f-b337-a72aa1a7c41b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:52.910985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31df3-93d0-4a5f-b337-a72aa1a7c41b service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:52.911302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31df3-93d0-4a5f-b337-a72aa1a7c41b service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:52.911588 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31df3-93d0-4a5f-b337-a72aa1a7c41b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:52.925096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec31df3-93d0-4a5f-b337-a72aa1a7c41b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:52.926586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec31df3-93d0-4a5f-b337-a72aa1a7c41b service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:52.938532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1109/2243] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:23:53.276264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee95d7-4bed-4e42-b0a4-aa16e515e37d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:53.279489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-21ee95d7-4bed-4e42-b0a4-aa16e515e37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:53.296844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee95d7-4bed-4e42-b0a4-aa16e515e37d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:53.303056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:53.304864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:53.305508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:53.305947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] PATH_INFO: `/v3/projects/0c6c7256c24245f192960e6df9211c29` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:53.308502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6c7256c24245f192960e6df9211c29)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:53.310840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:53.394095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6c7256c24245f192960e6df9211c29', 'request_id': 'req-9a52abf1-829b-41aa-a56e-8bcfc2630c93'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:53.402044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6c7256c24245f192960e6df9211c29', 'request_id': 'req-9a52abf1-829b-41aa-a56e-8bcfc2630c93'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:53.418485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6c7256c24245f192960e6df9211c29', 'request_id': 'req-9a52abf1-829b-41aa-a56e-8bcfc2630c93'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:53.435806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] The token cache is being invalidate because project 0c6c7256c24245f192960e6df9211c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:23:53.436213 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:53.436634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a52abf1-829b-41aa-a56e-8bcfc2630c93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:53.438476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1135/2244] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:23:53 2022] DELETE /identity/v3/projects/0c6c7256c24245f192960e6df9211c29 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:23:53.443816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24125976-fd82-4e26-a623-b57886c79c67 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:53.444386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24125976-fd82-4e26-a623-b57886c79c67 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:53.444386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24125976-fd82-4e26-a623-b57886c79c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:53.496956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-24125976-fd82-4e26-a623-b57886c79c67 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:53.498586 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-24125976-fd82-4e26-a623-b57886c79c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:53.568151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1110/2245] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:53.578132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca071528-06cf-4bb7-9674-76515fe89246 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:53.578470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca071528-06cf-4bb7-9674-76515fe89246 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:53.578831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca071528-06cf-4bb7-9674-76515fe89246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:53.609208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ca071528-06cf-4bb7-9674-76515fe89246 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:53.610818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ca071528-06cf-4bb7-9674-76515fe89246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:53.621133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1136/2246] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:53.624776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24125976-fd82-4e26-a623-b57886c79c67 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:53.638413 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24125976-fd82-4e26-a623-b57886c79c67 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:53.645134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:53.645980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:53.646265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:53.646537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:53.647315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:23:53.647626 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:53.649242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:53.676742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1111/2247] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:23:53.677618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca071528-06cf-4bb7-9674-76515fe89246 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:53.696679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca071528-06cf-4bb7-9674-76515fe89246 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:53.701753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427a12e-7cae-4777-ad76-3c8b0e603f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:53.702430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a12e-7cae-4777-ad76-3c8b0e603f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:53.702709 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a12e-7cae-4777-ad76-3c8b0e603f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:53.702976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a12e-7cae-4777-ad76-3c8b0e603f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:53.703717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427a12e-7cae-4777-ad76-3c8b0e603f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:53.705651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427a12e-7cae-4777-ad76-3c8b0e603f48 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:53.721638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:53.747504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1137/2248] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:23:53 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:23:53.751746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:53.768980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50408c9a-3a5c-4435-9bb5-52b9e0f4f32f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:53.882614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a1b84-c1c7-4629-9ebe-efd9be0868fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:53.883656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a1b84-c1c7-4629-9ebe-efd9be0868fe admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:53.883977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a1b84-c1c7-4629-9ebe-efd9be0868fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:53.884377 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a1b84-c1c7-4629-9ebe-efd9be0868fe admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:53.885150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a1b84-c1c7-4629-9ebe-efd9be0868fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:53.887688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a1b84-c1c7-4629-9ebe-efd9be0868fe admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:53.911061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1112/2249] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:23:53 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:23:53.915295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427a12e-7cae-4777-ad76-3c8b0e603f48 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:53.926437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427a12e-7cae-4777-ad76-3c8b0e603f48 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:53.931997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604a75-67c3-4d3e-81eb-f179f77ac857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:53.932674 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604a75-67c3-4d3e-81eb-f179f77ac857 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:53.932953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604a75-67c3-4d3e-81eb-f179f77ac857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:53.933393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604a75-67c3-4d3e-81eb-f179f77ac857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:53.934148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f604a75-67c3-4d3e-81eb-f179f77ac857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:53.935862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f604a75-67c3-4d3e-81eb-f179f77ac857 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:53.951334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1138/2250] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:23:53.953197 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a1b84-c1c7-4629-9ebe-efd9be0868fe admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:53.964132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a1b84-c1c7-4629-9ebe-efd9be0868fe admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:53.968924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fbf79-1046-4696-a615-5769a12b6045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:53.970678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fbf79-1046-4696-a615-5769a12b6045 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:53.970970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fbf79-1046-4696-a615-5769a12b6045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:53.971299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fbf79-1046-4696-a615-5769a12b6045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:53.972097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fbf79-1046-4696-a615-5769a12b6045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:53.973778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fbf79-1046-4696-a615-5769a12b6045 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:53.989638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1113/2251] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:23:53.995246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604a75-67c3-4d3e-81eb-f179f77ac857 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:54.016170 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604a75-67c3-4d3e-81eb-f179f77ac857 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:54.021812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:54.023040 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:54.023426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:54.023786 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] PATH_INFO: `/v3/projects/b2ecb25514c745939abff21b9503f7d7/users/6f336164927c40c0b388d07818e3a360/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:54.042802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ecb25514c745939abff21b9503f7d7, user_id=6f336164927c40c0b388d07818e3a360, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:54.045976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:54.071324 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1139/2252] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:23:53 2022] PUT /identity/v3/projects/b2ecb25514c745939abff21b9503f7d7/users/6f336164927c40c0b388d07818e3a360/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:23:54.081544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02055d7-93c1-4fbf-a25b-b7d93db8efb0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:54.081836 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02055d7-93c1-4fbf-a25b-b7d93db8efb0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:54.082110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02055d7-93c1-4fbf-a25b-b7d93db8efb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:54.111975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f02055d7-93c1-4fbf-a25b-b7d93db8efb0 None None] MFA Rules not processed for user `6f336164927c40c0b388d07818e3a360`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:54.113319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f02055d7-93c1-4fbf-a25b-b7d93db8efb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:54.127645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:54.195458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1114/2253] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:54.203829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:54.205645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:54.269747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda661b6-2ac1-48b3-a035-0661db9e7ebc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:54.273829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de5fd1-5706-4cc0-bf9c-f49c980fc8e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:54.274500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5fd1-5706-4cc0-bf9c-f49c980fc8e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:54.274788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5fd1-5706-4cc0-bf9c-f49c980fc8e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:54.275107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5fd1-5706-4cc0-bf9c-f49c980fc8e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:54.285247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de5fd1-5706-4cc0-bf9c-f49c980fc8e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:54.287135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de5fd1-5706-4cc0-bf9c-f49c980fc8e0 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:54.300182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1140/2254] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:23:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:23:57.171042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02055d7-93c1-4fbf-a25b-b7d93db8efb0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:57.244151 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02055d7-93c1-4fbf-a25b-b7d93db8efb0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:57.249034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:57.249953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:57.250232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:57.250561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] PATH_INFO: `/v3/users/6f336164927c40c0b388d07818e3a360` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:57.251725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f336164927c40c0b388d07818e3a360)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:57.253672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:57.305822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f336164927c40c0b388d07818e3a360'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.313510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f336164927c40c0b388d07818e3a360'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.320532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f336164927c40c0b388d07818e3a360'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.321171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f336164927c40c0b388d07818e3a360'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.334057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f336164927c40c0b388d07818e3a360'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.340065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1115/2255] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:23:57 2022] DELETE /identity/v3/users/6f336164927c40c0b388d07818e3a360 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:23:57.345469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5c15b-a89e-4878-9f7f-b3f08a94b6bc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:57.346023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5c15b-a89e-4878-9f7f-b3f08a94b6bc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:57.346023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5c15b-a89e-4878-9f7f-b3f08a94b6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:57.369534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1de5c15b-a89e-4878-9f7f-b3f08a94b6bc None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:57.370768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1de5c15b-a89e-4878-9f7f-b3f08a94b6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:57.431890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1141/2256] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:57.441522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:57.442988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:57.478327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:57.505395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c2e75-b1c3-4c4f-9f11-169baf1f9907 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:57.509495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d99480-dfd3-43d8-a912-5203dfde326d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:57.510158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d99480-dfd3-43d8-a912-5203dfde326d service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:57.510442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d99480-dfd3-43d8-a912-5203dfde326d service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:57.510718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d99480-dfd3-43d8-a912-5203dfde326d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:57.521355 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d99480-dfd3-43d8-a912-5203dfde326d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:57.522863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d99480-dfd3-43d8-a912-5203dfde326d service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:57.533021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1116/2257] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:23:57.807644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5c15b-a89e-4878-9f7f-b3f08a94b6bc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:57.809377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1de5c15b-a89e-4878-9f7f-b3f08a94b6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:57.824462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5c15b-a89e-4878-9f7f-b3f08a94b6bc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:57.829923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:57.830860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:57.831143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:57.832105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] PATH_INFO: `/v3/projects/b2ecb25514c745939abff21b9503f7d7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:57.833572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ecb25514c745939abff21b9503f7d7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:57.835740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:57.886845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ecb25514c745939abff21b9503f7d7', 'request_id': 'req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.891385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:57.892654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ecb25514c745939abff21b9503f7d7', 'request_id': 'req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.902202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ecb25514c745939abff21b9503f7d7', 'request_id': 'req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.913453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] The token cache is being invalidate because project b2ecb25514c745939abff21b9503f7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:23:57.913741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.914385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-659b4737-0ada-4e0c-a3a2-0fc2e53e9eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:23:57.916183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1142/2258] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:23:57 2022] DELETE /identity/v3/projects/b2ecb25514c745939abff21b9503f7d7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:23:57.921570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef39ee-7d2a-438e-8a9a-0044c8be059a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:57.921844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef39ee-7d2a-438e-8a9a-0044c8be059a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:57.922127 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef39ee-7d2a-438e-8a9a-0044c8be059a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:57.941694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ebef39ee-7d2a-438e-8a9a-0044c8be059a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:57.942891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ebef39ee-7d2a-438e-8a9a-0044c8be059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:58.002126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1117/2259] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:58.010479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5a8f7-b236-44dc-bb52-e58d1f48e52d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.010775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5a8f7-b236-44dc-bb52-e58d1f48e52d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.011054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5a8f7-b236-44dc-bb52-e58d1f48e52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.031155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-58b5a8f7-b236-44dc-bb52-e58d1f48e52d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:58.032477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58b5a8f7-b236-44dc-bb52-e58d1f48e52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:58.042196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1143/2260] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:58.047336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef39ee-7d2a-438e-8a9a-0044c8be059a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:58.061865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef39ee-7d2a-438e-8a9a-0044c8be059a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:58.069372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:58.069993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.070301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.070613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.071387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:23:58.071716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:58.073384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:58.085894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1118/2261] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:23:58.090671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5a8f7-b236-44dc-bb52-e58d1f48e52d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:58.100149 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5a8f7-b236-44dc-bb52-e58d1f48e52d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:58.105272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13404fa5-4ff4-47c0-9763-fe820d15e60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:58.105984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404fa5-4ff4-47c0-9763-fe820d15e60e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.106266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404fa5-4ff4-47c0-9763-fe820d15e60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.106540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404fa5-4ff4-47c0-9763-fe820d15e60e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.107266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13404fa5-4ff4-47c0-9763-fe820d15e60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:58.109460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13404fa5-4ff4-47c0-9763-fe820d15e60e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:58.137840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1144/2262] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:23:58 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:23:58.142415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:58.153380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b017d19-6766-4b5b-9e2f-de08941384ce admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:58.213033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93f62f-a38a-45f1-98ea-df869d0b9bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:58.213761 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93f62f-a38a-45f1-98ea-df869d0b9bec admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.213995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93f62f-a38a-45f1-98ea-df869d0b9bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.214260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93f62f-a38a-45f1-98ea-df869d0b9bec admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.214964 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93f62f-a38a-45f1-98ea-df869d0b9bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:58.217002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93f62f-a38a-45f1-98ea-df869d0b9bec admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:58.241033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1119/2263] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:23:58 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:23:58.244440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13404fa5-4ff4-47c0-9763-fe820d15e60e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:58.256267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13404fa5-4ff4-47c0-9763-fe820d15e60e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:58.261555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff57d4d-a168-4765-9c83-e72ad54ae31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:58.262243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57d4d-a168-4765-9c83-e72ad54ae31d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.262527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57d4d-a168-4765-9c83-e72ad54ae31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.262801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57d4d-a168-4765-9c83-e72ad54ae31d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.263604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff57d4d-a168-4765-9c83-e72ad54ae31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:58.265229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff57d4d-a168-4765-9c83-e72ad54ae31d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:58.277091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1145/2264] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:23:58.282087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93f62f-a38a-45f1-98ea-df869d0b9bec admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:58.293914 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93f62f-a38a-45f1-98ea-df869d0b9bec admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:58.299647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe82f49-2508-4c50-8550-112620147288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:58.300434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe82f49-2508-4c50-8550-112620147288 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.300805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe82f49-2508-4c50-8550-112620147288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.301178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe82f49-2508-4c50-8550-112620147288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.302016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe82f49-2508-4c50-8550-112620147288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:58.303890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe82f49-2508-4c50-8550-112620147288 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:58.321878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1120/2265] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:23:58.323166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff57d4d-a168-4765-9c83-e72ad54ae31d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:58.334146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff57d4d-a168-4765-9c83-e72ad54ae31d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:58.339823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:58.340935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.341420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.342303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] PATH_INFO: `/v3/projects/b75a43658cad40baa25b533455161a29/users/e0c5a3d6bff2493d9bc34082dc6787f0/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.361436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75a43658cad40baa25b533455161a29, user_id=e0c5a3d6bff2493d9bc34082dc6787f0, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:58.364393 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:58.391256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1146/2266] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:23:58 2022] PUT /identity/v3/projects/b75a43658cad40baa25b533455161a29/users/e0c5a3d6bff2493d9bc34082dc6787f0/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:23:58.402775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806fa3-c772-4381-a8f5-ee77b332a1ac None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.403550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806fa3-c772-4381-a8f5-ee77b332a1ac None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.403550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806fa3-c772-4381-a8f5-ee77b332a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.431981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ad806fa3-c772-4381-a8f5-ee77b332a1ac None None] MFA Rules not processed for user `e0c5a3d6bff2493d9bc34082dc6787f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:58.433057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ad806fa3-c772-4381-a8f5-ee77b332a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:58.470693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:58.509664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1121/2267] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:23:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:58.521626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:58.523569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:58.588900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088870b-59a3-496e-b7f2-5f72b4ae1672 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:58.593316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:58.593969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.594246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.594514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.603639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:58.605071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:58.614524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1147/2268] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:23:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:23:58.667631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0c3b4-8d7c-4a2b-8260-2d1738e61567 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:58.668013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0c3b4-8d7c-4a2b-8260-2d1738e61567 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:58.668519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0c3b4-8d7c-4a2b-8260-2d1738e61567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:58.721753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5cf0c3b4-8d7c-4a2b-8260-2d1738e61567 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:23:58.722971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5cf0c3b4-8d7c-4a2b-8260-2d1738e61567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:58.799369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1122/2269] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 08:23:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:23:58.808981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:23:58.810665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:23:58.999833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f979b-10ac-4597-b052-1fce60d52b95 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:23:59.005248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef68b1-a747-4f5a-a968-116fdc6afb33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:23:59.005907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef68b1-a747-4f5a-a968-116fdc6afb33 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:23:59.006190 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef68b1-a747-4f5a-a968-116fdc6afb33 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:23:59.006465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef68b1-a747-4f5a-a968-116fdc6afb33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:23:59.012459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:23:59.018257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef68b1-a747-4f5a-a968-116fdc6afb33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:23:59.020111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef68b1-a747-4f5a-a968-116fdc6afb33 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:23:59.035391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1148/2270] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:23:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 08:24:01.933653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ee13-9ce0-4ead-b8bb-ebfa0aef72db None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:01.934122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ee13-9ce0-4ead-b8bb-ebfa0aef72db None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:01.934201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ee13-9ce0-4ead-b8bb-ebfa0aef72db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:01.955951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8d78ee13-9ce0-4ead-b8bb-ebfa0aef72db None None] MFA Rules not processed for user `938efacddf794fd18cd323043906b6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:01.957624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8d78ee13-9ce0-4ead-b8bb-ebfa0aef72db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:01.965922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f911-7dcc-407b-9ad0-723d64b2ea9e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:01.966808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f911-7dcc-407b-9ad0-723d64b2ea9e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:01.967142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f911-7dcc-407b-9ad0-723d64b2ea9e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:01.968951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1149/2271] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:24:01.995845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39eab-bcb9-47ab-aa54-f48e8a7a6711 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:01.996133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39eab-bcb9-47ab-aa54-f48e8a7a6711 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:01.996771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39eab-bcb9-47ab-aa54-f48e8a7a6711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:02.043986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1123/2272] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:02.076848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3df39eab-bcb9-47ab-aa54-f48e8a7a6711 None None] MFA Rules not processed for user `b93c07dd32f543a69e74599297cbe17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:02.078111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3df39eab-bcb9-47ab-aa54-f48e8a7a6711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:02.147288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78ee13-9ce0-4ead-b8bb-ebfa0aef72db None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:02.149000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8d78ee13-9ce0-4ead-b8bb-ebfa0aef72db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:02.254245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1150/2273] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:24:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:02.292997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df39eab-bcb9-47ab-aa54-f48e8a7a6711 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:02.295558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3df39eab-bcb9-47ab-aa54-f48e8a7a6711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:02.296030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:02.320109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df39eab-bcb9-47ab-aa54-f48e8a7a6711 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:02.329339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78ee13-9ce0-4ead-b8bb-ebfa0aef72db None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:02.335835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:02.336715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:02.337002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:02.337342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] PATH_INFO: `/v3/users/2821263e860a449eb4472ed2c4203e20` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:02.338698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2821263e860a449eb4472ed2c4203e20)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:02.340880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:02.341391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78b509-8a5c-4fa4-b4b5-3d639fb0738a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:02.342075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78b509-8a5c-4fa4-b4b5-3d639fb0738a service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:02.342358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78b509-8a5c-4fa4-b4b5-3d639fb0738a service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:02.342745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78b509-8a5c-4fa4-b4b5-3d639fb0738a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:02.369576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78b509-8a5c-4fa4-b4b5-3d639fb0738a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:02.373111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78b509-8a5c-4fa4-b4b5-3d639fb0738a service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:02.386878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1151/2274] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:24:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:24:02.395596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2821263e860a449eb4472ed2c4203e20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.407557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2821263e860a449eb4472ed2c4203e20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.418226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2821263e860a449eb4472ed2c4203e20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.418963 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2821263e860a449eb4472ed2c4203e20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.436889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-458aa97e-e809-446a-b352-c567d04cff2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2821263e860a449eb4472ed2c4203e20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.443920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1124/2275] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:24:02 2022] DELETE /identity/v3/users/2821263e860a449eb4472ed2c4203e20 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:24:02.811090 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78b509-8a5c-4fa4-b4b5-3d639fb0738a service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:02.823258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78b509-8a5c-4fa4-b4b5-3d639fb0738a service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:02.885797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:02.886677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:02.886956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:02.887255 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] PATH_INFO: `/v3/projects/ad9e87cb4bc84ff294c69c2e54704c33` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:02.888433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9e87cb4bc84ff294c69c2e54704c33)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:02.890354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:02.947878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9e87cb4bc84ff294c69c2e54704c33', 'request_id': 'req-2f0d3ffa-501b-43a4-a22b-0a881b612521'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.954062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9e87cb4bc84ff294c69c2e54704c33', 'request_id': 'req-2f0d3ffa-501b-43a4-a22b-0a881b612521'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.966426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9e87cb4bc84ff294c69c2e54704c33', 'request_id': 'req-2f0d3ffa-501b-43a4-a22b-0a881b612521'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.967717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:02.984924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] The token cache is being invalidate because project ad9e87cb4bc84ff294c69c2e54704c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:24:02.985385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.986207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f0d3ffa-501b-43a4-a22b-0a881b612521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:02.988106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1152/2276] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:24:02 2022] DELETE /identity/v3/projects/ad9e87cb4bc84ff294c69c2e54704c33 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:02.992992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f7764-3b63-4dcb-a518-25580c4248ef None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:02.993511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f7764-3b63-4dcb-a518-25580c4248ef None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:02.993637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f7764-3b63-4dcb-a518-25580c4248ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.014204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e99f7764-3b63-4dcb-a518-25580c4248ef None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:03.015456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e99f7764-3b63-4dcb-a518-25580c4248ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:03.077442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1125/2277] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:24:03.085002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05129-feb0-46a3-997f-41fbf4fa5e7c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.085292 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05129-feb0-46a3-997f-41fbf4fa5e7c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.085592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05129-feb0-46a3-997f-41fbf4fa5e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.104378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2ff05129-feb0-46a3-997f-41fbf4fa5e7c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:03.105576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2ff05129-feb0-46a3-997f-41fbf4fa5e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:03.114477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1153/2278] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:03.118782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f7764-3b63-4dcb-a518-25580c4248ef None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:03.128675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f7764-3b63-4dcb-a518-25580c4248ef None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:03.135485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:03.136170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.136494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.136815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.138092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:24:03.138228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:03.140268 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:03.154649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1126/2279] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:24:03.158853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05129-feb0-46a3-997f-41fbf4fa5e7c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:03.170526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05129-feb0-46a3-997f-41fbf4fa5e7c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:03.176735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999c61b-8620-45ad-a0ca-69a38dbabcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:03.177635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999c61b-8620-45ad-a0ca-69a38dbabcae admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.178078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999c61b-8620-45ad-a0ca-69a38dbabcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.178516 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999c61b-8620-45ad-a0ca-69a38dbabcae admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.179257 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999c61b-8620-45ad-a0ca-69a38dbabcae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:03.193908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999c61b-8620-45ad-a0ca-69a38dbabcae admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:03.242780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1154/2280] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:24:03 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:24:03.247534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:03.258111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ed011-ec3b-4619-ba47-157d09adc8ba admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:03.321653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:03.332160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900a0e7-c651-43db-aac0-1671fb66ddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:03.333388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a0e7-c651-43db-aac0-1671fb66ddcb admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.333737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a0e7-c651-43db-aac0-1671fb66ddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.334049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a0e7-c651-43db-aac0-1671fb66ddcb admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.334799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900a0e7-c651-43db-aac0-1671fb66ddcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:03.337334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900a0e7-c651-43db-aac0-1671fb66ddcb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:03.365695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1127/2281] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:24:03 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:24:03.368031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999c61b-8620-45ad-a0ca-69a38dbabcae admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:03.380508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999c61b-8620-45ad-a0ca-69a38dbabcae admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:03.388451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216f50-a748-4fab-9f1d-7187f23ecb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:03.388739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216f50-a748-4fab-9f1d-7187f23ecb1b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.389269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216f50-a748-4fab-9f1d-7187f23ecb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.389269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216f50-a748-4fab-9f1d-7187f23ecb1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.390072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23216f50-a748-4fab-9f1d-7187f23ecb1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:03.391840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23216f50-a748-4fab-9f1d-7187f23ecb1b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:03.404411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1155/2282] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:24:03.408552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900a0e7-c651-43db-aac0-1671fb66ddcb admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:03.420613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900a0e7-c651-43db-aac0-1671fb66ddcb admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:03.429199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144fb10-a688-4b56-8965-38c6257dfe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:03.430271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144fb10-a688-4b56-8965-38c6257dfe12 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.430679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144fb10-a688-4b56-8965-38c6257dfe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.431482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144fb10-a688-4b56-8965-38c6257dfe12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.433531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144fb10-a688-4b56-8965-38c6257dfe12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:03.434159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144fb10-a688-4b56-8965-38c6257dfe12 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:03.446843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1128/2283] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:24:03.451093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216f50-a748-4fab-9f1d-7187f23ecb1b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:03.464287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216f50-a748-4fab-9f1d-7187f23ecb1b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:03.471017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:03.472087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.472388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.472717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] PATH_INFO: `/v3/projects/27ae014ce2e0415bb630752610a02fda/users/ca09e984a7a9494b9910bbc4d632b194/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.515073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ae014ce2e0415bb630752610a02fda, user_id=ca09e984a7a9494b9910bbc4d632b194, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:03.518647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:03.537885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1156/2284] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:24:03 2022] PUT /identity/v3/projects/27ae014ce2e0415bb630752610a02fda/users/ca09e984a7a9494b9910bbc4d632b194/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:24:03.547530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58003d2-9c68-4ddd-a0c5-ac367749d9cb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.547843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58003d2-9c68-4ddd-a0c5-ac367749d9cb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.548125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58003d2-9c68-4ddd-a0c5-ac367749d9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.575258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c58003d2-9c68-4ddd-a0c5-ac367749d9cb None None] MFA Rules not processed for user `ca09e984a7a9494b9910bbc4d632b194`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:03.576400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c58003d2-9c68-4ddd-a0c5-ac367749d9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:03.643991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1129/2285] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:03.655572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:03.657390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:03.719660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bdc6a-7bbe-4d80-b11d-a833de93708c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:03.723792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b5ad7-400e-40ef-9f52-3c61cbd52f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:03.724440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b5ad7-400e-40ef-9f52-3c61cbd52f67 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.724717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b5ad7-400e-40ef-9f52-3c61cbd52f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.724990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b5ad7-400e-40ef-9f52-3c61cbd52f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.734085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b5ad7-400e-40ef-9f52-3c61cbd52f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:03.735576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b5ad7-400e-40ef-9f52-3c61cbd52f67 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:03.744379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1157/2286] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:24:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:24:03.776838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58003d2-9c68-4ddd-a0c5-ac367749d9cb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:03.845210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58003d2-9c68-4ddd-a0c5-ac367749d9cb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:03.850930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:03.852384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.852987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.853332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] PATH_INFO: `/v3/users/ca09e984a7a9494b9910bbc4d632b194` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.855442 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca09e984a7a9494b9910bbc4d632b194)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:03.858617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:03.918751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca09e984a7a9494b9910bbc4d632b194'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:03.932469 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca09e984a7a9494b9910bbc4d632b194'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:03.940441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca09e984a7a9494b9910bbc4d632b194'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:03.941138 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca09e984a7a9494b9910bbc4d632b194'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:03.943090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:03.958881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca09e984a7a9494b9910bbc4d632b194'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:03.967072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1130/2287] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:24:03 2022] DELETE /identity/v3/users/ca09e984a7a9494b9910bbc4d632b194 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:03.971690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c73bfc-fb35-4221-8e21-1054d16f9598 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:03.972173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c73bfc-fb35-4221-8e21-1054d16f9598 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:03.972298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c73bfc-fb35-4221-8e21-1054d16f9598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:03.991440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-29c73bfc-fb35-4221-8e21-1054d16f9598 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:03.992690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-29c73bfc-fb35-4221-8e21-1054d16f9598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:04.062758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1158/2288] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:04.072432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:04.073971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:04.139756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4e1ba-2473-464a-b17b-840bd6607d3f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:04.143317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cc5a8-2f9e-403d-af44-01645d43c162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:04.143965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc5a8-2f9e-403d-af44-01645d43c162 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:04.144539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc5a8-2f9e-403d-af44-01645d43c162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:04.144839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc5a8-2f9e-403d-af44-01645d43c162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:04.154096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cc5a8-2f9e-403d-af44-01645d43c162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:04.155566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cc5a8-2f9e-403d-af44-01645d43c162 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:04.166057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1131/2289] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:24:04.501423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c73bfc-fb35-4221-8e21-1054d16f9598 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:04.503168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-29c73bfc-fb35-4221-8e21-1054d16f9598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:04.535873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c73bfc-fb35-4221-8e21-1054d16f9598 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:04.552283 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:04.553672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:04.554035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:04.554439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] PATH_INFO: `/v3/projects/27ae014ce2e0415bb630752610a02fda` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:04.556321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ae014ce2e0415bb630752610a02fda)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:04.561233 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:04.601417 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:04.640137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ae014ce2e0415bb630752610a02fda', 'request_id': 'req-96079691-6eb1-486a-933c-32a749ccce62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:04.647784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ae014ce2e0415bb630752610a02fda', 'request_id': 'req-96079691-6eb1-486a-933c-32a749ccce62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:04.661202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ae014ce2e0415bb630752610a02fda', 'request_id': 'req-96079691-6eb1-486a-933c-32a749ccce62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:04.678360 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] The token cache is being invalidate because project 27ae014ce2e0415bb630752610a02fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:24:04.678812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:04.679615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-96079691-6eb1-486a-933c-32a749ccce62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:04.682235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1159/2290] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:24:04 2022] DELETE /identity/v3/projects/27ae014ce2e0415bb630752610a02fda => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:04.688511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830f8c8-d514-4538-b7e4-168e53d791f1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:04.688511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830f8c8-d514-4538-b7e4-168e53d791f1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:04.688997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830f8c8-d514-4538-b7e4-168e53d791f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:04.727201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b830f8c8-d514-4538-b7e4-168e53d791f1 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:04.729170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b830f8c8-d514-4538-b7e4-168e53d791f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:04.850782 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1132/2291] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:04.859726 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee1f9c-c470-44a8-81ce-5cf4c824712d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:04.860020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee1f9c-c470-44a8-81ce-5cf4c824712d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:04.860298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee1f9c-c470-44a8-81ce-5cf4c824712d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:04.886987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ccee1f9c-c470-44a8-81ce-5cf4c824712d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:04.888405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ccee1f9c-c470-44a8-81ce-5cf4c824712d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:04.905371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1160/2292] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:24:04.906699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830f8c8-d514-4538-b7e4-168e53d791f1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:04.929133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830f8c8-d514-4538-b7e4-168e53d791f1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:04.934213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:04.935715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:04.936001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:04.936280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:04.937905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:24:04.938219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:04.941663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:04.955860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1133/2293] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:24:04.960208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee1f9c-c470-44a8-81ce-5cf4c824712d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:04.972809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee1f9c-c470-44a8-81ce-5cf4c824712d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:04.977159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916bc0b-676a-45ab-80ad-04c4e18e2058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:04.978131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916bc0b-676a-45ab-80ad-04c4e18e2058 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:04.978484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916bc0b-676a-45ab-80ad-04c4e18e2058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:04.978890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916bc0b-676a-45ab-80ad-04c4e18e2058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:04.979747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916bc0b-676a-45ab-80ad-04c4e18e2058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:04.981755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916bc0b-676a-45ab-80ad-04c4e18e2058 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:05.015241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1161/2294] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:24:04 2022] POST /identity/v3/projects => generated 397 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:24:05.017495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:05.029490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0791c7-a13d-4e74-bff3-090d53cd1b9f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:05.093029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39e511-7db9-42d4-8ae7-0a33c378f4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:05.093785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39e511-7db9-42d4-8ae7-0a33c378f4c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:05.094028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39e511-7db9-42d4-8ae7-0a33c378f4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:05.094330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39e511-7db9-42d4-8ae7-0a33c378f4c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:05.095108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39e511-7db9-42d4-8ae7-0a33c378f4c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:05.097387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39e511-7db9-42d4-8ae7-0a33c378f4c3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:05.122754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1134/2295] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:24:05 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:24:05.124237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916bc0b-676a-45ab-80ad-04c4e18e2058 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:05.134900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916bc0b-676a-45ab-80ad-04c4e18e2058 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:05.143367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0af9ec-adc9-44f8-b4f6-78545ee194e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:05.145182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0af9ec-adc9-44f8-b4f6-78545ee194e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:05.145597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0af9ec-adc9-44f8-b4f6-78545ee194e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:05.145974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0af9ec-adc9-44f8-b4f6-78545ee194e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:05.147139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0af9ec-adc9-44f8-b4f6-78545ee194e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:05.148887 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0af9ec-adc9-44f8-b4f6-78545ee194e3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:05.164645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1162/2296] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:24:05.167504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39e511-7db9-42d4-8ae7-0a33c378f4c3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:05.172839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:05.178796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39e511-7db9-42d4-8ae7-0a33c378f4c3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:05.185047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6891-4d6e-40bc-b2f3-10442611d3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:05.191851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6891-4d6e-40bc-b2f3-10442611d3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:05.191968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6891-4d6e-40bc-b2f3-10442611d3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:05.192359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6891-4d6e-40bc-b2f3-10442611d3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:05.194224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6891-4d6e-40bc-b2f3-10442611d3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:05.196398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6891-4d6e-40bc-b2f3-10442611d3b3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:05.214581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1135/2297] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:24:05.218648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0af9ec-adc9-44f8-b4f6-78545ee194e3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:05.235902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0af9ec-adc9-44f8-b4f6-78545ee194e3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:05.241167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:05.242256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:05.242660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:05.243035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] PATH_INFO: `/v3/projects/31e2444ec7044d1894b66fd3c2987850/users/197e2c4d05a94781a0a27345f232deff/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:05.267629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e2444ec7044d1894b66fd3c2987850, user_id=197e2c4d05a94781a0a27345f232deff, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:05.270343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:05.285423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1163/2298] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:24:05 2022] PUT /identity/v3/projects/31e2444ec7044d1894b66fd3c2987850/users/197e2c4d05a94781a0a27345f232deff/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:05.296681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea55dce-64c7-4a48-b6b4-695a333e9f0e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:05.297050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea55dce-64c7-4a48-b6b4-695a333e9f0e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:05.297399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea55dce-64c7-4a48-b6b4-695a333e9f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:05.328406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6ea55dce-64c7-4a48-b6b4-695a333e9f0e None None] MFA Rules not processed for user `197e2c4d05a94781a0a27345f232deff`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:05.329634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6ea55dce-64c7-4a48-b6b4-695a333e9f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:05.400904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1136/2299] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:24:05.412985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:05.415007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:05.475965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3c118-6681-4207-b274-ea0ea710b4e4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:05.479859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb5588-63b0-4704-9e99-a2b854280546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:05.480486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5588-63b0-4704-9e99-a2b854280546 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:05.480792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5588-63b0-4704-9e99-a2b854280546 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:05.481033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5588-63b0-4704-9e99-a2b854280546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:05.497746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb5588-63b0-4704-9e99-a2b854280546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:05.499201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb5588-63b0-4704-9e99-a2b854280546 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:05.515926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1164/2300] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:24:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:24:15.397324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=36 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:15.510108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:25.048155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea55dce-64c7-4a48-b6b4-695a333e9f0e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:25.124104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea55dce-64c7-4a48-b6b4-695a333e9f0e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:25.131555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:25.132868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:25.133280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:25.133642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] PATH_INFO: `/v3/users/197e2c4d05a94781a0a27345f232deff` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:25.135602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=197e2c4d05a94781a0a27345f232deff)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:25.138582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:25.191868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '197e2c4d05a94781a0a27345f232deff'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.200884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '197e2c4d05a94781a0a27345f232deff'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.208239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '197e2c4d05a94781a0a27345f232deff'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.208871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '197e2c4d05a94781a0a27345f232deff'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.220646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '197e2c4d05a94781a0a27345f232deff'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.226854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1137/2301] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:24:25 2022] DELETE /identity/v3/users/197e2c4d05a94781a0a27345f232deff => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:25.231455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191cde0-fa5a-49a8-bad6-6031a8f049ab None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:25.231928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191cde0-fa5a-49a8-bad6-6031a8f049ab None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:25.232021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191cde0-fa5a-49a8-bad6-6031a8f049ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:25.249286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f191cde0-fa5a-49a8-bad6-6031a8f049ab None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:25.250468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f191cde0-fa5a-49a8-bad6-6031a8f049ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:25.312613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1165/2302] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:25.322412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:25.324280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:25.385425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be003d-8cd5-4e49-9d66-ea537a8264a0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:25.390646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc7178-de96-497e-a9f0-a2eaa2b61242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:25.391478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc7178-de96-497e-a9f0-a2eaa2b61242 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:25.391604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc7178-de96-497e-a9f0-a2eaa2b61242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:25.391872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc7178-de96-497e-a9f0-a2eaa2b61242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:25.403241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc7178-de96-497e-a9f0-a2eaa2b61242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:25.404732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc7178-de96-497e-a9f0-a2eaa2b61242 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:25.414630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1138/2303] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:24:25.778554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191cde0-fa5a-49a8-bad6-6031a8f049ab None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:25.780236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f191cde0-fa5a-49a8-bad6-6031a8f049ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:25.798552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191cde0-fa5a-49a8-bad6-6031a8f049ab None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:25.803608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:25.804487 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:25.804765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:25.805041 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] PATH_INFO: `/v3/projects/31e2444ec7044d1894b66fd3c2987850` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:25.806414 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e2444ec7044d1894b66fd3c2987850)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:25.808636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:25.857063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e2444ec7044d1894b66fd3c2987850', 'request_id': 'req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.863452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e2444ec7044d1894b66fd3c2987850', 'request_id': 'req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.872945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e2444ec7044d1894b66fd3c2987850', 'request_id': 'req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.885812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] The token cache is being invalidate because project 31e2444ec7044d1894b66fd3c2987850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:24:25.886099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.886862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-55ecc0f7-5ae6-4122-9c8b-39d383f61a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:25.888601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1166/2304] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:24:25 2022] DELETE /identity/v3/projects/31e2444ec7044d1894b66fd3c2987850 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:30.018789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07303-1f33-4b0b-8c21-27f9d6e3968d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:30.019384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07303-1f33-4b0b-8c21-27f9d6e3968d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:30.019384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07303-1f33-4b0b-8c21-27f9d6e3968d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:30.038432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-45f07303-1f33-4b0b-8c21-27f9d6e3968d None None] MFA Rules not processed for user `44b66024bd78487082154441343a2161`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:30.039685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-45f07303-1f33-4b0b-8c21-27f9d6e3968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:30.058112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:30.101821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1139/2305] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:35.882760 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:39.542650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45844f43-b15b-41b4-b26a-bad4a05e1178 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:39.543539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45844f43-b15b-41b4-b26a-bad4a05e1178 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:39.543539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45844f43-b15b-41b4-b26a-bad4a05e1178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:39.562519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-45844f43-b15b-41b4-b26a-bad4a05e1178 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:39.563780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45844f43-b15b-41b4-b26a-bad4a05e1178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:39.628493 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1167/2306] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:24:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:24:39.651354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f07303-1f33-4b0b-8c21-27f9d6e3968d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:39.653110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-45f07303-1f33-4b0b-8c21-27f9d6e3968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:39.725895 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f07303-1f33-4b0b-8c21-27f9d6e3968d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:39.738056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596a577-026e-428a-a466-c3988d56aec4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:39.738732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a577-026e-428a-a466-c3988d56aec4 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:39.739016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a577-026e-428a-a466-c3988d56aec4 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:39.739330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a577-026e-428a-a466-c3988d56aec4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:39.749707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596a577-026e-428a-a466-c3988d56aec4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:39.751272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596a577-026e-428a-a466-c3988d56aec4 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:39.760557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1140/2307] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:24:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:24:40.327402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4718f4-e882-40f3-9dea-e730f0b096d4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:40.327697 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4718f4-e882-40f3-9dea-e730f0b096d4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:40.327980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4718f4-e882-40f3-9dea-e730f0b096d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:40.348445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fc4718f4-e882-40f3-9dea-e730f0b096d4 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:40.349736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fc4718f4-e882-40f3-9dea-e730f0b096d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:40.357520 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1168/2308] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 08:24:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:40.370339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596a577-026e-428a-a466-c3988d56aec4 service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:40.381831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596a577-026e-428a-a466-c3988d56aec4 service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:40.388842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804734f5-48b7-40a8-8523-4f18e361125c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:40.389842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804734f5-48b7-40a8-8523-4f18e361125c service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:40.390274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804734f5-48b7-40a8-8523-4f18e361125c service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:40.390697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804734f5-48b7-40a8-8523-4f18e361125c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:40.401245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804734f5-48b7-40a8-8523-4f18e361125c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:40.403337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804734f5-48b7-40a8-8523-4f18e361125c service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:40.412942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1141/2309] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:24:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 08:24:40.663155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22807e15-e4d6-4648-9285-76831c2b188e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:40.663695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22807e15-e4d6-4648-9285-76831c2b188e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:40.663804 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22807e15-e4d6-4648-9285-76831c2b188e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:40.681819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-22807e15-e4d6-4648-9285-76831c2b188e None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:40.683053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-22807e15-e4d6-4648-9285-76831c2b188e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:40.691174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1169/2310] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:24:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:40.703867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804734f5-48b7-40a8-8523-4f18e361125c service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:40.719794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804734f5-48b7-40a8-8523-4f18e361125c service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:40.728619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb8043-7b5e-4f99-bd25-ff7cc6e3cd39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:40.729566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb8043-7b5e-4f99-bd25-ff7cc6e3cd39 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:40.729946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb8043-7b5e-4f99-bd25-ff7cc6e3cd39 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:40.730349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb8043-7b5e-4f99-bd25-ff7cc6e3cd39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:40.738799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb8043-7b5e-4f99-bd25-ff7cc6e3cd39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:40.740430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb8043-7b5e-4f99-bd25-ff7cc6e3cd39 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:40.749048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1142/2311] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:24:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:24:41.087950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908138-5558-42a7-ac19-1ac84d1d8450 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:41.088236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908138-5558-42a7-ac19-1ac84d1d8450 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:41.088715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908138-5558-42a7-ac19-1ac84d1d8450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:41.110209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b4908138-5558-42a7-ac19-1ac84d1d8450 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:41.111531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b4908138-5558-42a7-ac19-1ac84d1d8450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:41.119091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1170/2312] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 08:24:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:41.132361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb8043-7b5e-4f99-bd25-ff7cc6e3cd39 service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:41.146098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb8043-7b5e-4f99-bd25-ff7cc6e3cd39 service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:41.152532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef202d7-f424-4e1e-bf9a-c658319dfc02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:41.152770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef202d7-f424-4e1e-bf9a-c658319dfc02 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:41.153052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef202d7-f424-4e1e-bf9a-c658319dfc02 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:41.153336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef202d7-f424-4e1e-bf9a-c658319dfc02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:41.163133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef202d7-f424-4e1e-bf9a-c658319dfc02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:41.164771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef202d7-f424-4e1e-bf9a-c658319dfc02 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:41.173564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1143/2313] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:24:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 08:24:44.402337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4908138-5558-42a7-ac19-1ac84d1d8450 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:44.405019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b4908138-5558-42a7-ac19-1ac84d1d8450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:44.480036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4908138-5558-42a7-ac19-1ac84d1d8450 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:44.492521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:44.493438 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:44.493715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:44.493990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] PATH_INFO: `/v3/users/578195adbe574c67a6de17a17513f545` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:44.495948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=578195adbe574c67a6de17a17513f545)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:44.498074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:44.551474 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578195adbe574c67a6de17a17513f545'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:44.552132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578195adbe574c67a6de17a17513f545'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:44.559427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578195adbe574c67a6de17a17513f545'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:44.572749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578195adbe574c67a6de17a17513f545'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:44.582250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578195adbe574c67a6de17a17513f545'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:44.589169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1171/2314] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:24:44 2022] DELETE /identity/v3/users/578195adbe574c67a6de17a17513f545 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:44.594167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0823da-d980-44dc-b39e-101ad5596154 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:44.594735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0823da-d980-44dc-b39e-101ad5596154 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:44.594856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0823da-d980-44dc-b39e-101ad5596154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:44.614288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bf0823da-d980-44dc-b39e-101ad5596154 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:44.615568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bf0823da-d980-44dc-b39e-101ad5596154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:44.683372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1144/2315] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:44.693857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:44.695089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:44.702496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:44.775067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089ad8f-5f52-4bc7-8336-e0f011ca5a56 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:44.779404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb339e9-5016-4665-a0ea-acdca09158fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:44.780957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb339e9-5016-4665-a0ea-acdca09158fb service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:44.781229 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb339e9-5016-4665-a0ea-acdca09158fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:44.781510 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb339e9-5016-4665-a0ea-acdca09158fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:44.796717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb339e9-5016-4665-a0ea-acdca09158fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:44.798375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb339e9-5016-4665-a0ea-acdca09158fb service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:44.811143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1172/2316] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:24:45.023099 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0823da-d980-44dc-b39e-101ad5596154 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:45.024771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bf0823da-d980-44dc-b39e-101ad5596154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:45.037289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0823da-d980-44dc-b39e-101ad5596154 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:45.042520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:45.043449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.043724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.043992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] PATH_INFO: `/v3/projects/bf86466e66274193b5c2b0fdf69884b5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.045880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf86466e66274193b5c2b0fdf69884b5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:45.047537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:45.061395 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:45.101647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf86466e66274193b5c2b0fdf69884b5', 'request_id': 'req-27946264-e2c2-4d68-896e-e2c6209ac172'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:45.107257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf86466e66274193b5c2b0fdf69884b5', 'request_id': 'req-27946264-e2c2-4d68-896e-e2c6209ac172'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:45.115691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf86466e66274193b5c2b0fdf69884b5', 'request_id': 'req-27946264-e2c2-4d68-896e-e2c6209ac172'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:45.130095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] The token cache is being invalidate because project bf86466e66274193b5c2b0fdf69884b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:24:45.130375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:45.131012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-27946264-e2c2-4d68-896e-e2c6209ac172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:24:45.133232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1145/2317] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:24:45 2022] DELETE /identity/v3/projects/bf86466e66274193b5c2b0fdf69884b5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:45.140004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2b7d-8d32-48bb-9718-8b4a2c9c218a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.140310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2b7d-8d32-48bb-9718-8b4a2c9c218a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.140622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2b7d-8d32-48bb-9718-8b4a2c9c218a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.160467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d62a2b7d-8d32-48bb-9718-8b4a2c9c218a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:45.161701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d62a2b7d-8d32-48bb-9718-8b4a2c9c218a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:45.221405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1173/2318] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:45.230742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6631da3-d763-433b-9914-e696ac28acb8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.231031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6631da3-d763-433b-9914-e696ac28acb8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.231334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6631da3-d763-433b-9914-e696ac28acb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.255717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c6631da3-d763-433b-9914-e696ac28acb8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:45.256936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c6631da3-d763-433b-9914-e696ac28acb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:45.265385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1146/2319] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:24:45.269871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a2b7d-8d32-48bb-9718-8b4a2c9c218a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:45.280541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a2b7d-8d32-48bb-9718-8b4a2c9c218a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:45.285587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:45.286252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.286537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.286811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.287582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:24:45.287866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:45.289506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:45.301893 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1174/2320] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:24:45.306180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6631da3-d763-433b-9914-e696ac28acb8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:45.316467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6631da3-d763-433b-9914-e696ac28acb8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:45.321737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7920e-62e0-4f51-a40b-cad9a25a4688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:45.322443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7920e-62e0-4f51-a40b-cad9a25a4688 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.322728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7920e-62e0-4f51-a40b-cad9a25a4688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.323008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7920e-62e0-4f51-a40b-cad9a25a4688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.323766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7920e-62e0-4f51-a40b-cad9a25a4688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:45.325700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7920e-62e0-4f51-a40b-cad9a25a4688 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:45.367326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1147/2321] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:24:45 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:24:45.371958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:45.383143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2c46e-092f-4938-b92b-5a50f607c33b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:45.463471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41355d-309d-45f9-839d-9b33ee9df7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:45.464402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41355d-309d-45f9-839d-9b33ee9df7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.464733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41355d-309d-45f9-839d-9b33ee9df7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.465164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41355d-309d-45f9-839d-9b33ee9df7ae admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.466620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41355d-309d-45f9-839d-9b33ee9df7ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:45.468981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41355d-309d-45f9-839d-9b33ee9df7ae admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:45.492611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1175/2322] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:24:45 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:24:45.499432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7920e-62e0-4f51-a40b-cad9a25a4688 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:45.511412 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7920e-62e0-4f51-a40b-cad9a25a4688 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:45.517498 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24075-c291-4ec1-8235-ad29e391b531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:45.518190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24075-c291-4ec1-8235-ad29e391b531 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.518471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24075-c291-4ec1-8235-ad29e391b531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.518743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24075-c291-4ec1-8235-ad29e391b531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.519566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b24075-c291-4ec1-8235-ad29e391b531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:45.521184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b24075-c291-4ec1-8235-ad29e391b531 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:45.532042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1148/2323] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:24:45.536274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41355d-309d-45f9-839d-9b33ee9df7ae admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:45.548809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41355d-309d-45f9-839d-9b33ee9df7ae admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:45.555750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070895c-6a59-40f5-9694-9ee1128fee8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:45.556549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070895c-6a59-40f5-9694-9ee1128fee8d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.556897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070895c-6a59-40f5-9694-9ee1128fee8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.557169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070895c-6a59-40f5-9694-9ee1128fee8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.558015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070895c-6a59-40f5-9694-9ee1128fee8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:45.559784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070895c-6a59-40f5-9694-9ee1128fee8d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:45.571379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1176/2324] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:24:45.576036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24075-c291-4ec1-8235-ad29e391b531 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:45.587854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24075-c291-4ec1-8235-ad29e391b531 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:45.593580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:45.594490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.595118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.595118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] PATH_INFO: `/v3/projects/796208f488e742d1beaef6d02f0232e6/users/c685a84382ec4f4fbd3be4378ba14ca9/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.612358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796208f488e742d1beaef6d02f0232e6, user_id=c685a84382ec4f4fbd3be4378ba14ca9, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:45.615040 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:45.631608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1149/2325] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:24:45 2022] PUT /identity/v3/projects/796208f488e742d1beaef6d02f0232e6/users/c685a84382ec4f4fbd3be4378ba14ca9/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:45.636869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441509c-d612-4a80-86d6-b60093557621 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.637166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441509c-d612-4a80-86d6-b60093557621 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.637451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441509c-d612-4a80-86d6-b60093557621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.656047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8441509c-d612-4a80-86d6-b60093557621 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:45.657245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8441509c-d612-4a80-86d6-b60093557621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:45.666380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:45.723220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1177/2326] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:24:45.733149 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:45.735118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:45.812553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef89036-c174-45e9-b798-011c97b1d235 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:45.817002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aec77-66e7-4974-96ec-2f2cb666c45f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:45.817666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aec77-66e7-4974-96ec-2f2cb666c45f service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:45.817944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aec77-66e7-4974-96ec-2f2cb666c45f service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:45.818217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aec77-66e7-4974-96ec-2f2cb666c45f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:45.828171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801aec77-66e7-4974-96ec-2f2cb666c45f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:45.829608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801aec77-66e7-4974-96ec-2f2cb666c45f service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:45.839092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1150/2327] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:24:50.280976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3a345-bc40-47de-acd9-8bc1c1bad27a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:50.281486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3a345-bc40-47de-acd9-8bc1c1bad27a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:50.281594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3a345-bc40-47de-acd9-8bc1c1bad27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:50.342300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-abe3a345-bc40-47de-acd9-8bc1c1bad27a None None] MFA Rules not processed for user `c685a84382ec4f4fbd3be4378ba14ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:50.343731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-abe3a345-bc40-47de-acd9-8bc1c1bad27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:50.457036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1178/2328] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:50.461538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aec77-66e7-4974-96ec-2f2cb666c45f service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:50.474901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aec77-66e7-4974-96ec-2f2cb666c45f service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:50.480655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591227c5-e821-4857-85f8-63d8f7533392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:50.481332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591227c5-e821-4857-85f8-63d8f7533392 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:50.481615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591227c5-e821-4857-85f8-63d8f7533392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:50.481888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591227c5-e821-4857-85f8-63d8f7533392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:50.483098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591227c5-e821-4857-85f8-63d8f7533392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:50.488729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591227c5-e821-4857-85f8-63d8f7533392 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:50.523686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1151/2329] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:24:50 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:24:50.526646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3a345-bc40-47de-acd9-8bc1c1bad27a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:50.539162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3a345-bc40-47de-acd9-8bc1c1bad27a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:50.609769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3acc7f-2ac9-4763-9c38-30cdcddc35eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:50.610456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acc7f-2ac9-4763-9c38-30cdcddc35eb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:50.610739 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acc7f-2ac9-4763-9c38-30cdcddc35eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:50.611026 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acc7f-2ac9-4763-9c38-30cdcddc35eb admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:50.611839 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3acc7f-2ac9-4763-9c38-30cdcddc35eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:50.613796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3acc7f-2ac9-4763-9c38-30cdcddc35eb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:50.636670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1179/2330] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:24:50 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:24:50.641389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591227c5-e821-4857-85f8-63d8f7533392 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:50.652091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591227c5-e821-4857-85f8-63d8f7533392 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:50.657408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881906c0-16ec-4580-988b-2d4759fbd47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:50.658344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881906c0-16ec-4580-988b-2d4759fbd47b admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:50.658629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881906c0-16ec-4580-988b-2d4759fbd47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:50.658904 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881906c0-16ec-4580-988b-2d4759fbd47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:50.659696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881906c0-16ec-4580-988b-2d4759fbd47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:50.661321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881906c0-16ec-4580-988b-2d4759fbd47b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:50.673510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1152/2331] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:24:50.677812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3acc7f-2ac9-4763-9c38-30cdcddc35eb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:50.688651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3acc7f-2ac9-4763-9c38-30cdcddc35eb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:50.693618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485a4de-3494-446c-96cb-a7d8b9a319c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:50.694525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485a4de-3494-446c-96cb-a7d8b9a319c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:50.694801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485a4de-3494-446c-96cb-a7d8b9a319c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:50.695075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485a4de-3494-446c-96cb-a7d8b9a319c0 admin admin] PATH_INFO: `/v3/projects/267d08d441a5422687e94066bb5c35f7/users/e7f719eee8e04bf9bfb4ba1d67a62c16/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:50.711434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485a4de-3494-446c-96cb-a7d8b9a319c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=267d08d441a5422687e94066bb5c35f7, user_id=e7f719eee8e04bf9bfb4ba1d67a62c16, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:50.714096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485a4de-3494-446c-96cb-a7d8b9a319c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:50.728286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1180/2332] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:24:50 2022] PUT /identity/v3/projects/267d08d441a5422687e94066bb5c35f7/users/e7f719eee8e04bf9bfb4ba1d67a62c16/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:24:50.732815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881906c0-16ec-4580-988b-2d4759fbd47b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:50.743244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881906c0-16ec-4580-988b-2d4759fbd47b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:50.747958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:50.807739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a79e0-ef71-4afa-bf0f-70ad07d244bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:50.808506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a79e0-ef71-4afa-bf0f-70ad07d244bd admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:50.808854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a79e0-ef71-4afa-bf0f-70ad07d244bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:50.809192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a79e0-ef71-4afa-bf0f-70ad07d244bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:50.810002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a79e0-ef71-4afa-bf0f-70ad07d244bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:50.812047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a79e0-ef71-4afa-bf0f-70ad07d244bd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:50.823666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1153/2333] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:24:50.827771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485a4de-3494-446c-96cb-a7d8b9a319c0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:50.838405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485a4de-3494-446c-96cb-a7d8b9a319c0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:50.843337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:50.844296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:50.844569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:50.844836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] PATH_INFO: `/v3/projects/267d08d441a5422687e94066bb5c35f7/users/e7f719eee8e04bf9bfb4ba1d67a62c16/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:50.846785 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=267d08d441a5422687e94066bb5c35f7, user_id=e7f719eee8e04bf9bfb4ba1d67a62c16, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:50.849719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:50.863787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:24:50.868758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1181/2334] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:24:50 2022] PUT /identity/v3/projects/267d08d441a5422687e94066bb5c35f7/users/e7f719eee8e04bf9bfb4ba1d67a62c16/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:24:55.864336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a4af0-9517-4f7e-8d38-4a7f6b407f78 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:55.864913 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a4af0-9517-4f7e-8d38-4a7f6b407f78 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:55.864913 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a4af0-9517-4f7e-8d38-4a7f6b407f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:55.894449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-af2a4af0-9517-4f7e-8d38-4a7f6b407f78 None None] MFA Rules not processed for user `e7f719eee8e04bf9bfb4ba1d67a62c16`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:55.895723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-af2a4af0-9517-4f7e-8d38-4a7f6b407f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:55.969785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1154/2335] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:24:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:55.982484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:55.988137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:56.071984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc5cab-95c7-4738-b87b-ae10753f2108 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:56.075753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8cbcc9295074710aeacb35e4e1a4348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:56.076632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:56.076746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:56.076969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:56.135423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:56.136896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:56.148523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1182/2336] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:24:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:24:56.250746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4a8c-d8e8-4c76-b617-09a87d4249c2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:56.251282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4a8c-d8e8-4c76-b617-09a87d4249c2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:56.251566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4a8c-d8e8-4c76-b617-09a87d4249c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:56.271135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-902e4a8c-d8e8-4c76-b617-09a87d4249c2 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:56.272414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-902e4a8c-d8e8-4c76-b617-09a87d4249c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:56.344637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1155/2337] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:24:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:24:56.356881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:56.358934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:56.434834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9d2e7-7e22-4b00-9c26-5619b7e7bab9 service glance] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:56.441852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5d626-929a-4786-999c-4d091c1c1e25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:56.444005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d626-929a-4786-999c-4d091c1c1e25 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:56.444005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d626-929a-4786-999c-4d091c1c1e25 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:56.444005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d626-929a-4786-999c-4d091c1c1e25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:56.454328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5d626-929a-4786-999c-4d091c1c1e25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:56.455807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5d626-929a-4786-999c-4d091c1c1e25 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:56.465647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1183/2338] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:24:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:24:59.265625 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86306b0-adb1-4f5d-a4e3-5c966f4ffb85 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:59.266175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86306b0-adb1-4f5d-a4e3-5c966f4ffb85 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:59.266579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86306b0-adb1-4f5d-a4e3-5c966f4ffb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:59.291497 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d86306b0-adb1-4f5d-a4e3-5c966f4ffb85 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:24:59.293809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d86306b0-adb1-4f5d-a4e3-5c966f4ffb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:24:59.310287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1156/2339] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:24:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:24:59.325304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5d626-929a-4786-999c-4d091c1c1e25 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:24:59.338882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5d626-929a-4786-999c-4d091c1c1e25 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:24:59.346048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:24:59.346791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:24:59.347143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:24:59.347607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:24:59.357121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:24:59.359052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:24:59.370814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1184/2340] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:24:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:25:07.487832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a05ff-b6be-4011-a3ce-ae2a79143ad1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:07.488843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a05ff-b6be-4011-a3ce-ae2a79143ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:07.489296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a05ff-b6be-4011-a3ce-ae2a79143ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:07.521157 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e65a05ff-b6be-4011-a3ce-ae2a79143ad1 None None] MFA Rules not processed for user `c685a84382ec4f4fbd3be4378ba14ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:07.522777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e65a05ff-b6be-4011-a3ce-ae2a79143ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:07.530519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1157/2341] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:25:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:09.366769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:10.400610 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:10.402384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:10.469270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf16df-f623-49a7-973e-cbe4ce892f68 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:10.473328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:10.473999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:10.474288 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:10.474565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:10.476446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:10.585768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:10.587248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:10.607716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1185/2342] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:11.735058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08020-2150-452d-873c-65b9adc20f67 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:11.736014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08020-2150-452d-873c-65b9adc20f67 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:11.736458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08020-2150-452d-873c-65b9adc20f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:11.766266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f9d08020-2150-452d-873c-65b9adc20f67 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:11.767842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f9d08020-2150-452d-873c-65b9adc20f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:11.784425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1158/2343] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:25:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:11.798116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:11.811194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9001f-7979-4f4c-878f-ba182757e48b service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:11.818300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bac59-3e4b-4110-8b33-e3dd9e0c1194 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:11.819069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bac59-3e4b-4110-8b33-e3dd9e0c1194 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:11.819428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bac59-3e4b-4110-8b33-e3dd9e0c1194 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:11.819860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bac59-3e4b-4110-8b33-e3dd9e0c1194 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:11.830561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bac59-3e4b-4110-8b33-e3dd9e0c1194 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:11.831955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bac59-3e4b-4110-8b33-e3dd9e0c1194 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:11.841321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1186/2344] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:25:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:25:12.814241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d08020-2150-452d-873c-65b9adc20f67 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:12.815995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f9d08020-2150-452d-873c-65b9adc20f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:12.832891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:12.897048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d08020-2150-452d-873c-65b9adc20f67 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:12.901554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:12.902453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:12.902739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:12.903019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] PATH_INFO: `/v3/users/e0c5a3d6bff2493d9bc34082dc6787f0` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:12.904299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c5a3d6bff2493d9bc34082dc6787f0)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:12.906502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:12.918509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bac59-3e4b-4110-8b33-e3dd9e0c1194 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:12.928137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bac59-3e4b-4110-8b33-e3dd9e0c1194 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:12.932577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f918a-8d47-40d5-8cb0-94e0310ff618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:12.933375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f918a-8d47-40d5-8cb0-94e0310ff618 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:12.933678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f918a-8d47-40d5-8cb0-94e0310ff618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:12.934172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f918a-8d47-40d5-8cb0-94e0310ff618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:12.936537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-816f918a-8d47-40d5-8cb0-94e0310ff618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:12.956039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f918a-8d47-40d5-8cb0-94e0310ff618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:12.959326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f918a-8d47-40d5-8cb0-94e0310ff618 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:12.964304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c5a3d6bff2493d9bc34082dc6787f0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:12.973440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1187/2345] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:25:12.974094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c5a3d6bff2493d9bc34082dc6787f0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:12.981641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c5a3d6bff2493d9bc34082dc6787f0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:12.982330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c5a3d6bff2493d9bc34082dc6787f0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:12.996100 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c5a3d6bff2493d9bc34082dc6787f0'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:13.002735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1159/2346] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:25:12 2022] DELETE /identity/v3/users/e0c5a3d6bff2493d9bc34082dc6787f0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:25:13.007053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d64d4-7da0-4ec0-b675-2f995a6a5bb4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:13.007423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d64d4-7da0-4ec0-b675-2f995a6a5bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:13.007765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d64d4-7da0-4ec0-b675-2f995a6a5bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:13.028745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-223d64d4-7da0-4ec0-b675-2f995a6a5bb4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:13.030311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-223d64d4-7da0-4ec0-b675-2f995a6a5bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:13.085046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:13.101941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1188/2347] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:25:13.117729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:13.117729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:13.176992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3790f8-7188-4ba1-b4f5-757d298b7d0a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:13.180582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe80dae-b42f-4c51-9f19-ae3f14e45c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:13.181233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe80dae-b42f-4c51-9f19-ae3f14e45c43 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:13.181510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe80dae-b42f-4c51-9f19-ae3f14e45c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:13.181804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe80dae-b42f-4c51-9f19-ae3f14e45c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:13.190807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe80dae-b42f-4c51-9f19-ae3f14e45c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:13.192288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe80dae-b42f-4c51-9f19-ae3f14e45c43 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:13.202815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1160/2348] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:13.625045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d64d4-7da0-4ec0-b675-2f995a6a5bb4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:13.626811 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-223d64d4-7da0-4ec0-b675-2f995a6a5bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:13.640338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d64d4-7da0-4ec0-b675-2f995a6a5bb4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:13.649387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:13.649387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:13.649387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:13.649387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] PATH_INFO: `/v3/projects/b75a43658cad40baa25b533455161a29` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:13.651701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75a43658cad40baa25b533455161a29)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:13.653391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:13.703568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75a43658cad40baa25b533455161a29', 'request_id': 'req-cffcb588-db72-468b-ac8a-17c90c8ea5f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:13.710299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75a43658cad40baa25b533455161a29', 'request_id': 'req-cffcb588-db72-468b-ac8a-17c90c8ea5f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:13.721169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75a43658cad40baa25b533455161a29', 'request_id': 'req-cffcb588-db72-468b-ac8a-17c90c8ea5f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:13.735737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] The token cache is being invalidate because project b75a43658cad40baa25b533455161a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:25:13.736052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:13.736735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cffcb588-db72-468b-ac8a-17c90c8ea5f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:13.738370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1189/2349] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:25:13 2022] DELETE /identity/v3/projects/b75a43658cad40baa25b533455161a29 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:25:13.743338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3889-a0c2-4788-a53d-069f1660e869 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:13.743629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3889-a0c2-4788-a53d-069f1660e869 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:13.743905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3889-a0c2-4788-a53d-069f1660e869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:13.763698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-77fd3889-a0c2-4788-a53d-069f1660e869 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:13.765058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-77fd3889-a0c2-4788-a53d-069f1660e869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:13.796868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:13.828889 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1161/2350] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:13.836934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cce378-f9b7-437e-9aa8-7cdf7c5fb84f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:13.837224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cce378-f9b7-437e-9aa8-7cdf7c5fb84f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:13.837503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cce378-f9b7-437e-9aa8-7cdf7c5fb84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:13.856134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b8cce378-f9b7-437e-9aa8-7cdf7c5fb84f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:13.857424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b8cce378-f9b7-437e-9aa8-7cdf7c5fb84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:13.865663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1190/2351] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:25:13.869880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3889-a0c2-4788-a53d-069f1660e869 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:13.880426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3889-a0c2-4788-a53d-069f1660e869 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:13.886195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:13.886851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:13.887130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:13.887437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:13.888269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:25:13.888683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:13.890454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:13.903945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1162/2352] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:25:13.908517 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cce378-f9b7-437e-9aa8-7cdf7c5fb84f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:13.927021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cce378-f9b7-437e-9aa8-7cdf7c5fb84f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:13.932247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba86886-28da-4b67-8c3a-658c4081e8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:13.932944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba86886-28da-4b67-8c3a-658c4081e8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:13.933227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba86886-28da-4b67-8c3a-658c4081e8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:13.933558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba86886-28da-4b67-8c3a-658c4081e8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:13.934258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba86886-28da-4b67-8c3a-658c4081e8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:13.936950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba86886-28da-4b67-8c3a-658c4081e8e3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:13.966996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1191/2353] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:25:13 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:25:13.972075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:13.982232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e539d0-bc01-488e-8491-33b373fbc9cd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:14.042876 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afaf49-35c8-4574-92c8-0f40ffed2fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:14.043593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afaf49-35c8-4574-92c8-0f40ffed2fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:14.043900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afaf49-35c8-4574-92c8-0f40ffed2fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:14.044173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afaf49-35c8-4574-92c8-0f40ffed2fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:14.044932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afaf49-35c8-4574-92c8-0f40ffed2fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:14.046948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afaf49-35c8-4574-92c8-0f40ffed2fc0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:14.074652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1163/2354] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:25:13 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:25:14.079089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba86886-28da-4b67-8c3a-658c4081e8e3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:14.093119 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba86886-28da-4b67-8c3a-658c4081e8e3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:14.098357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe61d46-7c04-4566-9bcf-156fb0b30bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:14.099088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe61d46-7c04-4566-9bcf-156fb0b30bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:14.099406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe61d46-7c04-4566-9bcf-156fb0b30bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:14.099685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe61d46-7c04-4566-9bcf-156fb0b30bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:14.100435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe61d46-7c04-4566-9bcf-156fb0b30bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:14.102107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe61d46-7c04-4566-9bcf-156fb0b30bc5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:14.114103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1192/2355] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:25:14.119275 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afaf49-35c8-4574-92c8-0f40ffed2fc0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:14.130008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afaf49-35c8-4574-92c8-0f40ffed2fc0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:14.141161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bde29-28b6-4345-9e31-5c6f72cf522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:14.141956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bde29-28b6-4345-9e31-5c6f72cf522d admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:14.142311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bde29-28b6-4345-9e31-5c6f72cf522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:14.143162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bde29-28b6-4345-9e31-5c6f72cf522d admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:14.143517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bde29-28b6-4345-9e31-5c6f72cf522d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:14.145308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bde29-28b6-4345-9e31-5c6f72cf522d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:14.160945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1164/2356] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:25:14.162035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe61d46-7c04-4566-9bcf-156fb0b30bc5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:14.173834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe61d46-7c04-4566-9bcf-156fb0b30bc5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:14.179019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:14.179952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:14.180223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:14.180652 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] PATH_INFO: `/v3/projects/1da1b560a421476ba7f2204c34a33f22/users/85ea01236020490abae4cb038b7f783b/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:14.197074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da1b560a421476ba7f2204c34a33f22, user_id=85ea01236020490abae4cb038b7f783b, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:14.199755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:14.214879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1193/2357] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:25:14 2022] PUT /identity/v3/projects/1da1b560a421476ba7f2204c34a33f22/users/85ea01236020490abae4cb038b7f783b/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:25:14.226615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5bc30-c75b-4f45-87aa-5aafa2cdd686 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:14.227414 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5bc30-c75b-4f45-87aa-5aafa2cdd686 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:14.227921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5bc30-c75b-4f45-87aa-5aafa2cdd686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:14.258223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a4b5bc30-c75b-4f45-87aa-5aafa2cdd686 None None] MFA Rules not processed for user `85ea01236020490abae4cb038b7f783b`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:14.259622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a4b5bc30-c75b-4f45-87aa-5aafa2cdd686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:14.329719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1165/2358] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:25:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:14.341354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:14.343480 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:14.406534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d13059-ddd4-4294-8d99-1d7b6f292b7f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:14.410629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:14.411411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:14.411602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:14.411880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:14.422001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:14.423824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:14.434164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1194/2359] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:18.250526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5bc30-c75b-4f45-87aa-5aafa2cdd686 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:18.253237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a4b5bc30-c75b-4f45-87aa-5aafa2cdd686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:18.295018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:18.343723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5bc30-c75b-4f45-87aa-5aafa2cdd686 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:18.350300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:18.350962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:18.351248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:18.351555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:18.353862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:18.415894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:18.417430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:18.426778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1166/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:24.429103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:28.422577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=42 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:36.403556 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:36.405648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:36.406948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:36.408543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:36.474446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9e32e-e6bc-46c9-b65e-f090df76bc9e service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:36.474811 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e39f-3187-4776-aee3-d2d81d95b636 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:36.475835 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:36.476486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:36.476768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:36.477162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:36.479409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:36.479784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e70a9-f80a-4e1f-b379-e951bd60385f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:36.480445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e70a9-f80a-4e1f-b379-e951bd60385f service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:36.480726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e70a9-f80a-4e1f-b379-e951bd60385f service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:36.480986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e70a9-f80a-4e1f-b379-e951bd60385f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:36.482759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dd8e70a9-f80a-4e1f-b379-e951bd60385f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:36.489714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:36.491342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:36.499628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e70a9-f80a-4e1f-b379-e951bd60385f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:36.501122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1195/2361] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:36.501243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e70a9-f80a-4e1f-b379-e951bd60385f service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:36.511783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1167/2362] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:39.414629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:39.416192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:39.427597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819635e-acb5-4cbc-b39f-0b5f89ddda25 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:39.431617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45a3fc-3522-4d36-9e34-870d2bb2f9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:39.432260 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a3fc-3522-4d36-9e34-870d2bb2f9ef service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:39.432542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a3fc-3522-4d36-9e34-870d2bb2f9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:39.432818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a3fc-3522-4d36-9e34-870d2bb2f9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:39.434860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1b45a3fc-3522-4d36-9e34-870d2bb2f9ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:39.445526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45a3fc-3522-4d36-9e34-870d2bb2f9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:39.446986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45a3fc-3522-4d36-9e34-870d2bb2f9ef service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:39.456187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1196/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:25:39.574971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abe98b-9dbd-456d-aaa0-354cc9179507 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:39.576319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abe98b-9dbd-456d-aaa0-354cc9179507 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:39.576319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abe98b-9dbd-456d-aaa0-354cc9179507 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:39.577514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1168/2364] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:25:39.585462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d1d6-d69c-43a0-8a12-bc89f4b1bb78 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:39.585826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d1d6-d69c-43a0-8a12-bc89f4b1bb78 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:39.586179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d1d6-d69c-43a0-8a12-bc89f4b1bb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:39.641432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2c96d1d6-d69c-43a0-8a12-bc89f4b1bb78 None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:39.642765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c96d1d6-d69c-43a0-8a12-bc89f4b1bb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:39.658745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1197/2365] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:25:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:39.667129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abe98b-9dbd-456d-aaa0-354cc9179507 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:39.669789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-47abe98b-9dbd-456d-aaa0-354cc9179507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:39.734548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abe98b-9dbd-456d-aaa0-354cc9179507 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:39.738774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c9691-f416-473e-a931-8bf4db420ce1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8cbcc9295074710aeacb35e4e1a4348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:39.738965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9691-f416-473e-a931-8bf4db420ce1 service glance] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:39.739453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9691-f416-473e-a931-8bf4db420ce1 service glance] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:39.739883 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9691-f416-473e-a931-8bf4db420ce1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:39.751575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c9691-f416-473e-a931-8bf4db420ce1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:39.753016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c9691-f416-473e-a931-8bf4db420ce1 service glance] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:39.764616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1169/2366] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:39.838390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861349a5-68ce-4a8c-b049-035dcc3c8730 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:39.838609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861349a5-68ce-4a8c-b049-035dcc3c8730 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:39.838978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861349a5-68ce-4a8c-b049-035dcc3c8730 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:39.840527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1198/2367] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:25:39.846165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0bdc2-beda-4689-ad37-1db98b1cd173 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:39.846704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0bdc2-beda-4689-ad37-1db98b1cd173 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:39.847183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0bdc2-beda-4689-ad37-1db98b1cd173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:39.875183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a7d0bdc2-beda-4689-ad37-1db98b1cd173 None None] MFA Rules not processed for user `2965b1c8ab52457f8f167acda4ba6844`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:39.876681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a7d0bdc2-beda-4689-ad37-1db98b1cd173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:39.884509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1170/2368] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:25:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:39.897494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861349a5-68ce-4a8c-b049-035dcc3c8730 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:39.910384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861349a5-68ce-4a8c-b049-035dcc3c8730 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:39.914082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea783b3-83fb-4bc7-aab5-4fdae452b858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:39.914826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea783b3-83fb-4bc7-aab5-4fdae452b858 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:39.915168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea783b3-83fb-4bc7-aab5-4fdae452b858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:39.915540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea783b3-83fb-4bc7-aab5-4fdae452b858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:39.923864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea783b3-83fb-4bc7-aab5-4fdae452b858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:39.925332 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea783b3-83fb-4bc7-aab5-4fdae452b858 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:39.933432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1199/2369] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:40.027776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b91b3d-d820-46b0-b112-e76cc4e7c1d6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:40.028070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b91b3d-d820-46b0-b112-e76cc4e7c1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:40.028634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b91b3d-d820-46b0-b112-e76cc4e7c1d6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:40.030248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1171/2370] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 08:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 08:25:40.036101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2a194-cae0-4a4b-85c3-04c3cd56d102 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:40.036389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2a194-cae0-4a4b-85c3-04c3cd56d102 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:40.036667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2a194-cae0-4a4b-85c3-04c3cd56d102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:40.055633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-45a2a194-cae0-4a4b-85c3-04c3cd56d102 None None] MFA Rules not processed for user `96456bfed82d469b888bd495ec46e796`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:40.059808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45a2a194-cae0-4a4b-85c3-04c3cd56d102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:40.069573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1200/2371] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 08:25:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:40.080634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b91b3d-d820-46b0-b112-e76cc4e7c1d6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:40.092278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b91b3d-d820-46b0-b112-e76cc4e7c1d6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:40.095931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52367108-9181-4ac5-ab90-f388ff8b8215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:40.096575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367108-9181-4ac5-ab90-f388ff8b8215 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:40.096854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367108-9181-4ac5-ab90-f388ff8b8215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:40.097126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367108-9181-4ac5-ab90-f388ff8b8215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:40.107882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52367108-9181-4ac5-ab90-f388ff8b8215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:40.109308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52367108-9181-4ac5-ab90-f388ff8b8215 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:40.120403 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1172/2372] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:40.190075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2a194-cae0-4a4b-85c3-04c3cd56d102 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:40.191735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45a2a194-cae0-4a4b-85c3-04c3cd56d102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:40.266109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2a194-cae0-4a4b-85c3-04c3cd56d102 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:40.271082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b97f38-c80a-474d-86e0-d3d23101fffc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:40.271787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97f38-c80a-474d-86e0-d3d23101fffc service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:40.272064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97f38-c80a-474d-86e0-d3d23101fffc service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:40.272340 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97f38-c80a-474d-86e0-d3d23101fffc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:40.274379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f2b97f38-c80a-474d-86e0-d3d23101fffc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:40.287560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b97f38-c80a-474d-86e0-d3d23101fffc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:40.289221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b97f38-c80a-474d-86e0-d3d23101fffc service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:40.299660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1201/2373] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:25:43.386682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52367108-9181-4ac5-ab90-f388ff8b8215 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:43.450531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52367108-9181-4ac5-ab90-f388ff8b8215 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:43.455778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:43.456673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:43.457034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:43.457322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] PATH_INFO: `/v3/users/85ea01236020490abae4cb038b7f783b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:43.458644 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ea01236020490abae4cb038b7f783b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:43.460643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:43.493296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:43.519789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ea01236020490abae4cb038b7f783b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:43.528954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ea01236020490abae4cb038b7f783b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:43.537089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ea01236020490abae4cb038b7f783b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:43.537759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ea01236020490abae4cb038b7f783b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:43.550126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ea01236020490abae4cb038b7f783b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:43.556705 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1173/2374] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:25:43 2022] DELETE /identity/v3/users/85ea01236020490abae4cb038b7f783b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:25:43.561523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad88c-f2ac-4672-ac10-36161b7f4f4a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:43.562116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad88c-f2ac-4672-ac10-36161b7f4f4a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:43.562223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad88c-f2ac-4672-ac10-36161b7f4f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:43.582509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7bfad88c-f2ac-4672-ac10-36161b7f4f4a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:43.583660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7bfad88c-f2ac-4672-ac10-36161b7f4f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:43.620625 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:25:43.652923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1202/2375] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:43.664150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:43.665954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:43.729465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613d7c-1dd0-4818-9b7a-410555b9857c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:43.735091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac89d16-1eed-4175-b53f-f46f38e56c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:43.735877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89d16-1eed-4175-b53f-f46f38e56c89 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:43.736283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89d16-1eed-4175-b53f-f46f38e56c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:43.736545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89d16-1eed-4175-b53f-f46f38e56c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:43.747607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac89d16-1eed-4175-b53f-f46f38e56c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:43.749215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac89d16-1eed-4175-b53f-f46f38e56c89 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:43.760287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1174/2376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:25:44.058850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfad88c-f2ac-4672-ac10-36161b7f4f4a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:44.060376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7bfad88c-f2ac-4672-ac10-36161b7f4f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:44.078176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfad88c-f2ac-4672-ac10-36161b7f4f4a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:44.095877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847541-5978-4417-9a98-28acf231659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:44.097354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847541-5978-4417-9a98-28acf231659d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:44.098314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847541-5978-4417-9a98-28acf231659d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:44.098597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847541-5978-4417-9a98-28acf231659d admin admin] PATH_INFO: `/v3/projects/1da1b560a421476ba7f2204c34a33f22` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:44.100030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847541-5978-4417-9a98-28acf231659d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da1b560a421476ba7f2204c34a33f22)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:44.101959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847541-5978-4417-9a98-28acf231659d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:44.149504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71847541-5978-4417-9a98-28acf231659d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da1b560a421476ba7f2204c34a33f22', 'request_id': 'req-71847541-5978-4417-9a98-28acf231659d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:44.156987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71847541-5978-4417-9a98-28acf231659d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da1b560a421476ba7f2204c34a33f22', 'request_id': 'req-71847541-5978-4417-9a98-28acf231659d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:44.166600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71847541-5978-4417-9a98-28acf231659d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da1b560a421476ba7f2204c34a33f22', 'request_id': 'req-71847541-5978-4417-9a98-28acf231659d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:44.178479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71847541-5978-4417-9a98-28acf231659d admin admin] The token cache is being invalidate because project 1da1b560a421476ba7f2204c34a33f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:25:44.178763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71847541-5978-4417-9a98-28acf231659d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:44.179442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71847541-5978-4417-9a98-28acf231659d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:25:44.181069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1203/2377] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:25:44 2022] DELETE /identity/v3/projects/1da1b560a421476ba7f2204c34a33f22 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:25:51.749552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ccaaa-0d01-4f8f-be53-a8f1c28ceb82 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:51.750000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ccaaa-0d01-4f8f-be53-a8f1c28ceb82 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:51.750107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ccaaa-0d01-4f8f-be53-a8f1c28ceb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:51.769478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b50ccaaa-0d01-4f8f-be53-a8f1c28ceb82 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:25:51.770741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b50ccaaa-0d01-4f8f-be53-a8f1c28ceb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:51.834209 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1175/2378] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:25:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:25:51.856853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847541-5978-4417-9a98-28acf231659d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:25:51.858390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-71847541-5978-4417-9a98-28acf231659d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:25:51.930645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847541-5978-4417-9a98-28acf231659d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:25:51.936023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a697e-6def-45bf-b1f9-e44102303944 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:25:51.936675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a697e-6def-45bf-b1f9-e44102303944 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:25:51.936961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a697e-6def-45bf-b1f9-e44102303944 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:25:51.937239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a697e-6def-45bf-b1f9-e44102303944 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:25:51.946963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a697e-6def-45bf-b1f9-e44102303944 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:25:51.948662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a697e-6def-45bf-b1f9-e44102303944 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:25:51.958015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1204/2379] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:25:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:26:01.825773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=70,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:01.953731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:15.430160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ccaaa-0d01-4f8f-be53-a8f1c28ceb82 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:15.431767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b50ccaaa-0d01-4f8f-be53-a8f1c28ceb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:15.513136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ccaaa-0d01-4f8f-be53-a8f1c28ceb82 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:15.518725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:15.519675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:15.519957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:15.520237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] PATH_INFO: `/v3/users/44b66024bd78487082154441343a2161` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:15.521492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b66024bd78487082154441343a2161)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:15.523480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:15.574854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b66024bd78487082154441343a2161'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.585607 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b66024bd78487082154441343a2161'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.593148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b66024bd78487082154441343a2161'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.593826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b66024bd78487082154441343a2161'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.605857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-07682cd8-5e21-4f94-9de8-142695d03fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b66024bd78487082154441343a2161'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.613338 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1176/2380] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:26:15 2022] DELETE /identity/v3/users/44b66024bd78487082154441343a2161 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:26:15.786021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a697e-6def-45bf-b1f9-e44102303944 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:15.798526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a697e-6def-45bf-b1f9-e44102303944 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:15.857870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:15.858807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:15.859082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:15.859390 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] PATH_INFO: `/v3/projects/bea8f88d81c04623b66edaf1cebc2ee3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:15.860678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea8f88d81c04623b66edaf1cebc2ee3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:15.862728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:15.911014 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea8f88d81c04623b66edaf1cebc2ee3', 'request_id': 'req-b5a1c996-656b-4605-8a6e-89a65110609b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.916509 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea8f88d81c04623b66edaf1cebc2ee3', 'request_id': 'req-b5a1c996-656b-4605-8a6e-89a65110609b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.924791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea8f88d81c04623b66edaf1cebc2ee3', 'request_id': 'req-b5a1c996-656b-4605-8a6e-89a65110609b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.936719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] The token cache is being invalidate because project bea8f88d81c04623b66edaf1cebc2ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:26:15.937003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.937720 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5a1c996-656b-4605-8a6e-89a65110609b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:26:15.939401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1205/2381] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:26:15 2022] DELETE /identity/v3/projects/bea8f88d81c04623b66edaf1cebc2ee3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:26:15.944862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecb6c9-dfe5-4dc4-b2bb-074ff20de1f5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:15.945176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecb6c9-dfe5-4dc4-b2bb-074ff20de1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:15.945451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecb6c9-dfe5-4dc4-b2bb-074ff20de1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:15.964898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e6ecb6c9-dfe5-4dc4-b2bb-074ff20de1f5 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:26:15.966055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e6ecb6c9-dfe5-4dc4-b2bb-074ff20de1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:16.028017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1177/2382] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:26:16.091167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0e9cd-6807-4711-9dcd-04a0aaf4d7b0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.091497 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0e9cd-6807-4711-9dcd-04a0aaf4d7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.091775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0e9cd-6807-4711-9dcd-04a0aaf4d7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.115492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-baf0e9cd-6807-4711-9dcd-04a0aaf4d7b0 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:26:16.116705 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-baf0e9cd-6807-4711-9dcd-04a0aaf4d7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:16.126976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1206/2383] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:26:16.131619 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecb6c9-dfe5-4dc4-b2bb-074ff20de1f5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:16.142002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecb6c9-dfe5-4dc4-b2bb-074ff20de1f5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:16.148007 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:16.148664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.148939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.149208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.149926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:26:16.150230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:16.151878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:16.164304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1178/2384] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:26:16.169156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0e9cd-6807-4711-9dcd-04a0aaf4d7b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:16.179173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0e9cd-6807-4711-9dcd-04a0aaf4d7b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:16.184967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517b27-f9f6-4289-bdfd-4e83567792cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:16.185648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517b27-f9f6-4289-bdfd-4e83567792cb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.185922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517b27-f9f6-4289-bdfd-4e83567792cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.186280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517b27-f9f6-4289-bdfd-4e83567792cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.186998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80517b27-f9f6-4289-bdfd-4e83567792cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:16.188969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80517b27-f9f6-4289-bdfd-4e83567792cb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:16.220571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1207/2385] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:26:16 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:26:16.222548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:16.236141 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f75ba5-c6f9-420b-9373-6dce6a2852bf admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:16.239944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:16.324952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5257ed-2b1b-44a7-906e-74cbe094b287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:16.325671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5257ed-2b1b-44a7-906e-74cbe094b287 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.325955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5257ed-2b1b-44a7-906e-74cbe094b287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.327188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5257ed-2b1b-44a7-906e-74cbe094b287 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.329766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5257ed-2b1b-44a7-906e-74cbe094b287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:16.331825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5257ed-2b1b-44a7-906e-74cbe094b287 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:16.354443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1179/2386] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:26:16 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:26:16.358758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517b27-f9f6-4289-bdfd-4e83567792cb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:16.370582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517b27-f9f6-4289-bdfd-4e83567792cb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:16.376263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78105-78c2-4c6c-9acf-1d9c45f42b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:16.376997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78105-78c2-4c6c-9acf-1d9c45f42b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.377280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78105-78c2-4c6c-9acf-1d9c45f42b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.377552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78105-78c2-4c6c-9acf-1d9c45f42b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.378374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78105-78c2-4c6c-9acf-1d9c45f42b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:16.380044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78105-78c2-4c6c-9acf-1d9c45f42b5d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:16.391313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1208/2387] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:26:16.395480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5257ed-2b1b-44a7-906e-74cbe094b287 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:16.406371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5257ed-2b1b-44a7-906e-74cbe094b287 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:16.412486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab6a06-6615-43db-9bf5-c195fe10f053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:16.413233 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6a06-6615-43db-9bf5-c195fe10f053 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.413520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6a06-6615-43db-9bf5-c195fe10f053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.413795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6a06-6615-43db-9bf5-c195fe10f053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.414532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab6a06-6615-43db-9bf5-c195fe10f053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:16.416176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab6a06-6615-43db-9bf5-c195fe10f053 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:16.427260 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1180/2388] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:26:16.431483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78105-78c2-4c6c-9acf-1d9c45f42b5d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:16.442429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78105-78c2-4c6c-9acf-1d9c45f42b5d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:16.448365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:16.449278 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.449526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.449797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] PATH_INFO: `/v3/projects/aa6a422103704f6d982c2bc558b3387e/users/8d661ea344d74681bb7206f203033c60/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.464776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6a422103704f6d982c2bc558b3387e, user_id=8d661ea344d74681bb7206f203033c60, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:16.467642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:16.484466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1209/2389] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:26:16 2022] PUT /identity/v3/projects/aa6a422103704f6d982c2bc558b3387e/users/8d661ea344d74681bb7206f203033c60/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:26:16.489662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb3f4-ae81-4b57-b75e-8b349e701248 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.490136 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb3f4-ae81-4b57-b75e-8b349e701248 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.490230 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb3f4-ae81-4b57-b75e-8b349e701248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.509880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ae2cb3f4-ae81-4b57-b75e-8b349e701248 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:26:16.511115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ae2cb3f4-ae81-4b57-b75e-8b349e701248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:16.575694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1181/2390] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:26:16.585930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:16.587994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:16.620102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:16.653239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad03c-fbcc-4ef1-8a50-73a998612a7e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:16.657479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d866ea0-9600-4673-ab97-bf055d9a0c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:16.658146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d866ea0-9600-4673-ab97-bf055d9a0c78 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:16.658426 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d866ea0-9600-4673-ab97-bf055d9a0c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:16.658706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d866ea0-9600-4673-ab97-bf055d9a0c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:16.667833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d866ea0-9600-4673-ab97-bf055d9a0c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:16.669274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d866ea0-9600-4673-ab97-bf055d9a0c78 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:16.678812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1210/2391] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:26:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:26:20.883250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59cbf-6f50-4777-89f4-2c406cd6ffc1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:20.884108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59cbf-6f50-4777-89f4-2c406cd6ffc1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:20.884564 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59cbf-6f50-4777-89f4-2c406cd6ffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:20.914070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-78d59cbf-6f50-4777-89f4-2c406cd6ffc1 None None] MFA Rules not processed for user `8d661ea344d74681bb7206f203033c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:26:20.915935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-78d59cbf-6f50-4777-89f4-2c406cd6ffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:20.995511 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1182/2392] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:26:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:26:20.996721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d866ea0-9600-4673-ab97-bf055d9a0c78 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:21.040881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d866ea0-9600-4673-ab97-bf055d9a0c78 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:21.047939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321a9d1-7cf9-47b6-9b2e-d86f4c9f9017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:21.048900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321a9d1-7cf9-47b6-9b2e-d86f4c9f9017 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:21.049247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321a9d1-7cf9-47b6-9b2e-d86f4c9f9017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:21.049449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321a9d1-7cf9-47b6-9b2e-d86f4c9f9017 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:21.050134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321a9d1-7cf9-47b6-9b2e-d86f4c9f9017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:21.052353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321a9d1-7cf9-47b6-9b2e-d86f4c9f9017 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:21.091290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1211/2393] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:26:20 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:26:21.094787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59cbf-6f50-4777-89f4-2c406cd6ffc1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:21.106215 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59cbf-6f50-4777-89f4-2c406cd6ffc1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:21.124580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:21.192737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b811d6e-56d0-41b8-a60b-188831eb220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:21.193437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b811d6e-56d0-41b8-a60b-188831eb220d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:21.193716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b811d6e-56d0-41b8-a60b-188831eb220d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:21.194156 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b811d6e-56d0-41b8-a60b-188831eb220d admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:21.194859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b811d6e-56d0-41b8-a60b-188831eb220d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:21.196909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b811d6e-56d0-41b8-a60b-188831eb220d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:21.219882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1183/2394] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:26:21 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:26:21.224033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321a9d1-7cf9-47b6-9b2e-d86f4c9f9017 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:21.235189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321a9d1-7cf9-47b6-9b2e-d86f4c9f9017 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:21.240888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696c1b-9082-405b-8499-47f6923017a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:21.241587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696c1b-9082-405b-8499-47f6923017a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:21.241864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696c1b-9082-405b-8499-47f6923017a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:21.242136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696c1b-9082-405b-8499-47f6923017a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:21.242863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4696c1b-9082-405b-8499-47f6923017a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:21.244936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4696c1b-9082-405b-8499-47f6923017a4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:21.256563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1212/2395] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:26:21.261113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b811d6e-56d0-41b8-a60b-188831eb220d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:21.272023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b811d6e-56d0-41b8-a60b-188831eb220d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:21.276620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e84b0-e843-4721-848d-7fe296190a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:21.277561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e84b0-e843-4721-848d-7fe296190a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:21.277838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e84b0-e843-4721-848d-7fe296190a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:21.278117 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e84b0-e843-4721-848d-7fe296190a4f admin admin] PATH_INFO: `/v3/projects/598ad12efb0a4e5ab6a14de501494214/users/6c2aed93dad44ede9683370430c152a5/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:21.295663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e84b0-e843-4721-848d-7fe296190a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=598ad12efb0a4e5ab6a14de501494214, user_id=6c2aed93dad44ede9683370430c152a5, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:21.298363 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e84b0-e843-4721-848d-7fe296190a4f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:21.315200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1184/2396] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:26:21 2022] PUT /identity/v3/projects/598ad12efb0a4e5ab6a14de501494214/users/6c2aed93dad44ede9683370430c152a5/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:26:21.319386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696c1b-9082-405b-8499-47f6923017a4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:21.330836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696c1b-9082-405b-8499-47f6923017a4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:21.393685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:21.394371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:21.394703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:21.394974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:21.395732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:21.397366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:21.408545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1213/2397] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:26:21.412866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e84b0-e843-4721-848d-7fe296190a4f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:21.424227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e84b0-e843-4721-848d-7fe296190a4f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:21.429284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:21.430339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:21.430708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:21.431044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] PATH_INFO: `/v3/projects/598ad12efb0a4e5ab6a14de501494214/users/6c2aed93dad44ede9683370430c152a5/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:21.433279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=598ad12efb0a4e5ab6a14de501494214, user_id=6c2aed93dad44ede9683370430c152a5, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:21.436949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:21.449387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1185/2398] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:26:21 2022] PUT /identity/v3/projects/598ad12efb0a4e5ab6a14de501494214/users/6c2aed93dad44ede9683370430c152a5/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:26:22.140459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:22.143959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:22.220756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41902fec-b254-45da-8c0f-6c6d1957dd6e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:22.287329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:22.293105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:22.295171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:22.295612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:22.296074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:22.299002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:22.405638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:22.406467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:22.418626 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1214/2399] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:26:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:26:23.660216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:23.662155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:23.855566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1d18-75ce-4170-9de5-983d5707cc7b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:23.869983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207a7fa-f5a6-4402-8516-0d86ffeeaf84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:23.870672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207a7fa-f5a6-4402-8516-0d86ffeeaf84 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:23.871053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207a7fa-f5a6-4402-8516-0d86ffeeaf84 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:23.871231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207a7fa-f5a6-4402-8516-0d86ffeeaf84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:23.875166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c207a7fa-f5a6-4402-8516-0d86ffeeaf84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:23.891393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207a7fa-f5a6-4402-8516-0d86ffeeaf84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:23.893037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207a7fa-f5a6-4402-8516-0d86ffeeaf84 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:23.931741 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1186/2400] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:26:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:26:24.860156 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:24.887358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba08ef-2192-4fad-b0e1-e65cdefca29b service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:24.908003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:24.909737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:24.910147 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:24.910602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:24.917702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:24.959018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:24.961394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:24.990796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1215/2401] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:26:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:26:27.021716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0780db-1067-4343-903b-8456e18d2a2e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:27.022299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0780db-1067-4343-903b-8456e18d2a2e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:27.022509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0780db-1067-4343-903b-8456e18d2a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:27.056520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7f0780db-1067-4343-903b-8456e18d2a2e None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:26:27.062972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7f0780db-1067-4343-903b-8456e18d2a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:27.164880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:27.179605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1187/2402] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 08:26:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:26:27.201930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:27.203585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:27.315817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05590412-ca6b-41de-bdfc-64b8783a5af8 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:27.332960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4555e-71d8-45da-898d-61b9c0ce41b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:27.333649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4555e-71d8-45da-898d-61b9c0ce41b3 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:27.333925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4555e-71d8-45da-898d-61b9c0ce41b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:27.334201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4555e-71d8-45da-898d-61b9c0ce41b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:27.349457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4555e-71d8-45da-898d-61b9c0ce41b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:27.357707 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4555e-71d8-45da-898d-61b9c0ce41b3 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:27.385098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1216/2403] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:26:27.628647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a94dfb-8df2-49de-9388-3ff3623f8473 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:27.629184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a94dfb-8df2-49de-9388-3ff3623f8473 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:27.629581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a94dfb-8df2-49de-9388-3ff3623f8473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:27.670785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-10a94dfb-8df2-49de-9388-3ff3623f8473 None None] MFA Rules not processed for user `6c2aed93dad44ede9683370430c152a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:26:27.673525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-10a94dfb-8df2-49de-9388-3ff3623f8473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:27.824455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1188/2404] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:26:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:26:27.839006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4555e-71d8-45da-898d-61b9c0ce41b3 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:27.855003 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4555e-71d8-45da-898d-61b9c0ce41b3 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:27.869312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1193e3-b10a-4825-be55-efcb85995b7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:27.871576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1193e3-b10a-4825-be55-efcb85995b7b service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:27.875185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1193e3-b10a-4825-be55-efcb85995b7b service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:27.875663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1193e3-b10a-4825-be55-efcb85995b7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:28.006264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1193e3-b10a-4825-be55-efcb85995b7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:28.007975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1193e3-b10a-4825-be55-efcb85995b7b service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:28.024570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1217/2405] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:26:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:26:37.818176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=35 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:38.017828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:41.357041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a94dfb-8df2-49de-9388-3ff3623f8473 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:41.358901 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-10a94dfb-8df2-49de-9388-3ff3623f8473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:41.375543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a94dfb-8df2-49de-9388-3ff3623f8473 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:41.381574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:41.382224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:41.382509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:41.382785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:41.384874 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:41.450659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:41.452321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:41.462594 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1189/2406] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:26:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:26:51.457488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=28 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:26:54.625889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750e8c-179d-4e27-9c75-46d2a1275b91 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:54.626429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750e8c-179d-4e27-9c75-46d2a1275b91 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:54.626540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750e8c-179d-4e27-9c75-46d2a1275b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:54.645237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d7750e8c-179d-4e27-9c75-46d2a1275b91 None None] MFA Rules not processed for user `8d661ea344d74681bb7206f203033c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:26:54.646424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d7750e8c-179d-4e27-9c75-46d2a1275b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:54.653982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1218/2407] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:26:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:26:54.985332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:26:54.996492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724402f-552b-4056-887d-b894edfb2420 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:26:55.001258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b472c-9470-4df7-8115-feebe2de9411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:26:55.001882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b472c-9470-4df7-8115-feebe2de9411 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:26:55.002163 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b472c-9470-4df7-8115-feebe2de9411 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:26:55.002428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b472c-9470-4df7-8115-feebe2de9411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:26:55.004681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5a3b472c-9470-4df7-8115-feebe2de9411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:26:55.015103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b472c-9470-4df7-8115-feebe2de9411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:26:55.016590 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b472c-9470-4df7-8115-feebe2de9411 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:26:55.024922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1190/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:26:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:27:04.640573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:27:05.021327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:27:06.356818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7750e8c-179d-4e27-9c75-46d2a1275b91 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:27:06.371982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7750e8c-179d-4e27-9c75-46d2a1275b91 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:27:06.378911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:27:06.379601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:27:06.379883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:27:06.380140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:27:06.382246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:27:06.445745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:27:06.447387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:27:06.456926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1219/2409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:27:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:27:12.378556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b8ec-4fc6-4c12-a60c-8918993eec89 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:27:12.379162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b8ec-4fc6-4c12-a60c-8918993eec89 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:27:12.379162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b8ec-4fc6-4c12-a60c-8918993eec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:27:12.398391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c0c0b8ec-4fc6-4c12-a60c-8918993eec89 None None] MFA Rules not processed for user `a15a0abc2f25410b8476464d7cd32802`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:27:12.400249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c0c0b8ec-4fc6-4c12-a60c-8918993eec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:27:12.408031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1191/2410] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:27:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:27:13.647641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:27:13.657643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a836f5-ebbb-4685-b966-3ff6ab09ff60 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:27:13.667879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf980dd8-9201-42e0-88ec-8c96e96a2b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:27:13.668647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf980dd8-9201-42e0-88ec-8c96e96a2b43 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:27:13.669004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf980dd8-9201-42e0-88ec-8c96e96a2b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:27:13.669355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf980dd8-9201-42e0-88ec-8c96e96a2b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:27:13.680214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf980dd8-9201-42e0-88ec-8c96e96a2b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:27:13.681647 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf980dd8-9201-42e0-88ec-8c96e96a2b43 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:27:13.691932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1220/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:27:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:27:15.432305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b8ec-4fc6-4c12-a60c-8918993eec89 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:27:15.446108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b8ec-4fc6-4c12-a60c-8918993eec89 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:27:15.451787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848ecc4-bb57-49ec-833b-116d90712650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:27:15.452445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848ecc4-bb57-49ec-833b-116d90712650 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:27:15.452723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848ecc4-bb57-49ec-833b-116d90712650 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:27:15.452992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848ecc4-bb57-49ec-833b-116d90712650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:27:15.454773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2848ecc4-bb57-49ec-833b-116d90712650 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:27:15.468158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848ecc4-bb57-49ec-833b-116d90712650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:27:15.470112 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848ecc4-bb57-49ec-833b-116d90712650 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:27:15.482632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1192/2412] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:27:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:27:23.687790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=34 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:27:24.869437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec57d6-4d9d-47c9-980a-71935e88b757 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:27:24.870199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec57d6-4d9d-47c9-980a-71935e88b757 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:27:24.870199 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec57d6-4d9d-47c9-980a-71935e88b757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:27:24.887729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-33ec57d6-4d9d-47c9-980a-71935e88b757 None None] MFA Rules not processed for user `938efacddf794fd18cd323043906b6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:27:24.888917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-33ec57d6-4d9d-47c9-980a-71935e88b757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:27:24.896628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1221/2413] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:27:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:27:25.477342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=9 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:27:34.882829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:27:55.196571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f4c52-cee8-4f2c-9e26-08d8bb5809fc None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:27:55.197199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f4c52-cee8-4f2c-9e26-08d8bb5809fc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:27:55.197199 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f4c52-cee8-4f2c-9e26-08d8bb5809fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:27:55.219745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-327f4c52-cee8-4f2c-9e26-08d8bb5809fc None None] MFA Rules not processed for user `9b178568bc5e4876bdab388ac3302f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:27:55.220956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-327f4c52-cee8-4f2c-9e26-08d8bb5809fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:27:55.276817 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1193/2414] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:27:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:28:05.269858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=25 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:28:25.984101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec57d6-4d9d-47c9-980a-71935e88b757 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:25.985786 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-33ec57d6-4d9d-47c9-980a-71935e88b757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:26.059525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec57d6-4d9d-47c9-980a-71935e88b757 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:26.065167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:26.065809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:26.066087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:26.066354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:26.068343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:26.080640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:26.082232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:26.092659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1222/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:28:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:28:36.087962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=32 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:28:43.562873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f4c52-cee8-4f2c-9e26-08d8bb5809fc None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:43.575613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f4c52-cee8-4f2c-9e26-08d8bb5809fc None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:43.582244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:43.583142 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:43.583523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:43.583798 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:43.587001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:43.664411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:43.665868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:43.675815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1194/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:28:47.078554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:47.089758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3d6da-b7f5-499e-bfef-94a3373d5d0f service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:47.095162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:47.095902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.096154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.096628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.098457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:47.112907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:47.114873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:47.132721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:47.133195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1223/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:28:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:28:47.134404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:47.147764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d2b79-c452-4ab7-9471-9c1fa9937c93 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:47.154591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:47.155536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.155821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.156094 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] PATH_INFO: `/v3/users/a15a0abc2f25410b8476464d7cd32802` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.157488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15a0abc2f25410b8476464d7cd32802)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:47.159530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:47.208512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15a0abc2f25410b8476464d7cd32802'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.217326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15a0abc2f25410b8476464d7cd32802'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.224972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15a0abc2f25410b8476464d7cd32802'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.225612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15a0abc2f25410b8476464d7cd32802'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.238145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e6121f3c-fb29-4da1-b21a-801b73c4d2cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15a0abc2f25410b8476464d7cd32802'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.245017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1195/2418] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:28:47 2022] DELETE /identity/v3/users/a15a0abc2f25410b8476464d7cd32802 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:28:47.437191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:47.450214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3218d37-ab74-4268-80cd-3545d1997716 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:47.510604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:47.511541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.511825 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.512102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] PATH_INFO: `/v3/projects/d08f2f4ac6d54f7484711b28f48aeeb3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.513316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08f2f4ac6d54f7484711b28f48aeeb3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:47.515276 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:47.563885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08f2f4ac6d54f7484711b28f48aeeb3', 'request_id': 'req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.569303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08f2f4ac6d54f7484711b28f48aeeb3', 'request_id': 'req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.577831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08f2f4ac6d54f7484711b28f48aeeb3', 'request_id': 'req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.590495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] The token cache is being invalidate because project d08f2f4ac6d54f7484711b28f48aeeb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:28:47.590759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.591560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5f3bd83e-a8ad-4ff2-8c20-57d08b31feb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:47.593282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1224/2419] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:28:47 2022] DELETE /identity/v3/projects/d08f2f4ac6d54f7484711b28f48aeeb3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:28:47.598526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffbdee-4ef2-411c-a9ec-70af208c3122 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.598812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffbdee-4ef2-411c-a9ec-70af208c3122 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.599087 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffbdee-4ef2-411c-a9ec-70af208c3122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.618334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f9ffbdee-4ef2-411c-a9ec-70af208c3122 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:28:47.619553 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f9ffbdee-4ef2-411c-a9ec-70af208c3122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:47.682436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1196/2420] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:28:47.690752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8d4d-a907-4947-adf9-11e1874eb052 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.691044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8d4d-a907-4947-adf9-11e1874eb052 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.691362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8d4d-a907-4947-adf9-11e1874eb052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.709938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b0db8d4d-a907-4947-adf9-11e1874eb052 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:28:47.711183 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b0db8d4d-a907-4947-adf9-11e1874eb052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:47.720998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1225/2421] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:28:47.727438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffbdee-4ef2-411c-a9ec-70af208c3122 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:47.741649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffbdee-4ef2-411c-a9ec-70af208c3122 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:47.747009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:47.747706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.747984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.748250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.748950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:28:47.749251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:47.750856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:47.752472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:28:47.768892 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1197/2422] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:28:47.773330 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db8d4d-a907-4947-adf9-11e1874eb052 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:47.782939 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db8d4d-a907-4947-adf9-11e1874eb052 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:47.788502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef8bc-7ee3-4fc0-babb-372325bdb01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:47.789161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef8bc-7ee3-4fc0-babb-372325bdb01b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.789485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef8bc-7ee3-4fc0-babb-372325bdb01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.789758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef8bc-7ee3-4fc0-babb-372325bdb01b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.790460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ef8bc-7ee3-4fc0-babb-372325bdb01b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:47.792428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ef8bc-7ee3-4fc0-babb-372325bdb01b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:47.831891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1226/2423] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:28:47 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:28:47.836135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:47.847226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fcc33-bfa5-4a12-9fad-180b38bff452 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:47.909570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43695eb6-8c1c-48cd-a972-fc5f293b35b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:47.910254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695eb6-8c1c-48cd-a972-fc5f293b35b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.910536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695eb6-8c1c-48cd-a972-fc5f293b35b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.910795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695eb6-8c1c-48cd-a972-fc5f293b35b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.911659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43695eb6-8c1c-48cd-a972-fc5f293b35b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:47.914037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43695eb6-8c1c-48cd-a972-fc5f293b35b5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:47.939686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1198/2424] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:28:47 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:28:47.943967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef8bc-7ee3-4fc0-babb-372325bdb01b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:47.955459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef8bc-7ee3-4fc0-babb-372325bdb01b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:47.961655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6263e7-10f1-4e6f-a508-186d75e62762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:47.962322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6263e7-10f1-4e6f-a508-186d75e62762 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.962605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6263e7-10f1-4e6f-a508-186d75e62762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.962875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6263e7-10f1-4e6f-a508-186d75e62762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.963649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6263e7-10f1-4e6f-a508-186d75e62762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:47.965346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6263e7-10f1-4e6f-a508-186d75e62762 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:47.975873 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1227/2425] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:28:47.980147 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43695eb6-8c1c-48cd-a972-fc5f293b35b5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:47.990462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43695eb6-8c1c-48cd-a972-fc5f293b35b5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:47.995795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9469e-1ecc-4e90-8fa5-e15e6e681539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:47.996450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9469e-1ecc-4e90-8fa5-e15e6e681539 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:47.996748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9469e-1ecc-4e90-8fa5-e15e6e681539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:47.997020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9469e-1ecc-4e90-8fa5-e15e6e681539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:47.997745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9469e-1ecc-4e90-8fa5-e15e6e681539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:47.999354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9469e-1ecc-4e90-8fa5-e15e6e681539 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:48.009418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1199/2426] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:28:48.013778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6263e7-10f1-4e6f-a508-186d75e62762 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:48.025308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6263e7-10f1-4e6f-a508-186d75e62762 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:48.030439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:48.031394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:48.031752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:48.032076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] PATH_INFO: `/v3/projects/6aacebee42d84c2993866b60944cdf89/users/405f144a98c64c4cb0661c6ff3db9918/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:48.049911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aacebee42d84c2993866b60944cdf89, user_id=405f144a98c64c4cb0661c6ff3db9918, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:48.052756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:48.068446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1228/2427] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:28:48 2022] PUT /identity/v3/projects/6aacebee42d84c2993866b60944cdf89/users/405f144a98c64c4cb0661c6ff3db9918/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:28:48.079038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ddd35-8740-47b8-b39d-acd98a4c689d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:48.079359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ddd35-8740-47b8-b39d-acd98a4c689d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:48.079650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ddd35-8740-47b8-b39d-acd98a4c689d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:48.109026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3a9ddd35-8740-47b8-b39d-acd98a4c689d None None] MFA Rules not processed for user `405f144a98c64c4cb0661c6ff3db9918`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:28:48.110250 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3a9ddd35-8740-47b8-b39d-acd98a4c689d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:48.176464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1200/2428] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:28:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:28:48.188315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:48.190358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:48.197249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:28:48.266974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7b2b-f786-4318-ab55-aa98132a477c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:48.273212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d058e-ff29-4641-80d8-7ad41b5b773a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:48.273869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d058e-ff29-4641-80d8-7ad41b5b773a service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:48.274157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d058e-ff29-4641-80d8-7ad41b5b773a service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:48.274500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d058e-ff29-4641-80d8-7ad41b5b773a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:48.282972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d058e-ff29-4641-80d8-7ad41b5b773a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:48.284675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d058e-ff29-4641-80d8-7ad41b5b773a service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:48.294037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1229/2429] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:28:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:28:50.994435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ddd35-8740-47b8-b39d-acd98a4c689d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:51.064220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ddd35-8740-47b8-b39d-acd98a4c689d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:51.069847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:51.070744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:51.071023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:51.071348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] PATH_INFO: `/v3/users/405f144a98c64c4cb0661c6ff3db9918` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:51.072641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405f144a98c64c4cb0661c6ff3db9918)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:51.074633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:51.077334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:28:51.126631 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405f144a98c64c4cb0661c6ff3db9918'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.136329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405f144a98c64c4cb0661c6ff3db9918'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.144332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405f144a98c64c4cb0661c6ff3db9918'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.144967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405f144a98c64c4cb0661c6ff3db9918'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.156541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405f144a98c64c4cb0661c6ff3db9918'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.162968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1201/2430] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:28:50 2022] DELETE /identity/v3/users/405f144a98c64c4cb0661c6ff3db9918 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:28:51.167765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb980c4f-5199-4c11-8934-5204beb1d6e5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:51.168234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb980c4f-5199-4c11-8934-5204beb1d6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:51.168325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb980c4f-5199-4c11-8934-5204beb1d6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:51.186692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cb980c4f-5199-4c11-8934-5204beb1d6e5 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:28:51.187947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cb980c4f-5199-4c11-8934-5204beb1d6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:51.254242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1230/2431] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:28:51.264903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:51.267464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:51.326994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aed539-d218-4ae0-b39f-c3534cf73c75 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:51.330922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034b3dd-ca55-49f5-9591-dffb21b0b289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:51.331378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b3dd-ca55-49f5-9591-dffb21b0b289 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:51.331655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b3dd-ca55-49f5-9591-dffb21b0b289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:51.331926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b3dd-ca55-49f5-9591-dffb21b0b289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:51.341154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034b3dd-ca55-49f5-9591-dffb21b0b289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:51.342592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034b3dd-ca55-49f5-9591-dffb21b0b289 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:51.355055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1202/2432] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:28:51.640999 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb980c4f-5199-4c11-8934-5204beb1d6e5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:51.642455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cb980c4f-5199-4c11-8934-5204beb1d6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:51.655078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb980c4f-5199-4c11-8934-5204beb1d6e5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:51.661086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:51.661980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:51.662258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:51.662535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] PATH_INFO: `/v3/projects/6aacebee42d84c2993866b60944cdf89` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:51.663841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aacebee42d84c2993866b60944cdf89)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:51.666308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:51.717307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aacebee42d84c2993866b60944cdf89', 'request_id': 'req-3f18b3e2-c997-4243-8365-0d93a958a1d4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.722942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aacebee42d84c2993866b60944cdf89', 'request_id': 'req-3f18b3e2-c997-4243-8365-0d93a958a1d4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.731764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aacebee42d84c2993866b60944cdf89', 'request_id': 'req-3f18b3e2-c997-4243-8365-0d93a958a1d4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.743246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] The token cache is being invalidate because project 6aacebee42d84c2993866b60944cdf89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:28:51.743526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.744155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3f18b3e2-c997-4243-8365-0d93a958a1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:28:51.745710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1231/2433] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:28:51 2022] DELETE /identity/v3/projects/6aacebee42d84c2993866b60944cdf89 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:28:51.750940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17458114-c4de-4ea0-9320-a75b26fae370 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:51.751258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17458114-c4de-4ea0-9320-a75b26fae370 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:51.751537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17458114-c4de-4ea0-9320-a75b26fae370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:51.769985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-17458114-c4de-4ea0-9320-a75b26fae370 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:28:51.771248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-17458114-c4de-4ea0-9320-a75b26fae370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:51.832637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1203/2434] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:28:51.841096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abc391-4c17-40f4-a78b-4e3abf87ade4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:51.841383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abc391-4c17-40f4-a78b-4e3abf87ade4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:51.841653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abc391-4c17-40f4-a78b-4e3abf87ade4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:51.860402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-04abc391-4c17-40f4-a78b-4e3abf87ade4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:28:51.861691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-04abc391-4c17-40f4-a78b-4e3abf87ade4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:51.871511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1232/2435] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:28:51.876277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17458114-c4de-4ea0-9320-a75b26fae370 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:51.886830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17458114-c4de-4ea0-9320-a75b26fae370 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:51.892231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:51.892994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:51.893281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:51.893775 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:51.894526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:28:51.894839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:51.896507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:51.908152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1204/2436] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:28:51.912628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abc391-4c17-40f4-a78b-4e3abf87ade4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:51.921955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abc391-4c17-40f4-a78b-4e3abf87ade4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:51.926799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da1801-ede0-465c-963b-85f54be821f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:51.927479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da1801-ede0-465c-963b-85f54be821f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:51.927760 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da1801-ede0-465c-963b-85f54be821f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:51.928029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da1801-ede0-465c-963b-85f54be821f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:51.928733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da1801-ede0-465c-963b-85f54be821f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:51.930702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da1801-ede0-465c-963b-85f54be821f0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:51.934085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:28:51.960819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1233/2437] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:28:51 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:28:51.965125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:51.976301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a280dc-91fe-47aa-9ce6-974759abb8ef admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:51.985192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:28:52.038698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b2a38-1a28-488b-a896-148052f37551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:52.039400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b2a38-1a28-488b-a896-148052f37551 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:52.039682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b2a38-1a28-488b-a896-148052f37551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:52.039952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b2a38-1a28-488b-a896-148052f37551 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:52.040650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b2a38-1a28-488b-a896-148052f37551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:52.042589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b2a38-1a28-488b-a896-148052f37551 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:52.063864 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1205/2438] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:28:51 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:28:52.068185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da1801-ede0-465c-963b-85f54be821f0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:52.078826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da1801-ede0-465c-963b-85f54be821f0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:52.085088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ca31-e676-4b46-a3d0-abd2b8628abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:52.085764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ca31-e676-4b46-a3d0-abd2b8628abb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:52.086045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ca31-e676-4b46-a3d0-abd2b8628abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:52.086310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ca31-e676-4b46-a3d0-abd2b8628abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:52.087053 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780ca31-e676-4b46-a3d0-abd2b8628abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:52.088737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780ca31-e676-4b46-a3d0-abd2b8628abb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:52.099069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1234/2439] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:28:52.102905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b2a38-1a28-488b-a896-148052f37551 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:52.113652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b2a38-1a28-488b-a896-148052f37551 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:52.121054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da660f5-66d6-4d3f-a55c-2e4803f64530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:52.122104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da660f5-66d6-4d3f-a55c-2e4803f64530 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:52.122505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da660f5-66d6-4d3f-a55c-2e4803f64530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:52.122886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da660f5-66d6-4d3f-a55c-2e4803f64530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:52.124006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da660f5-66d6-4d3f-a55c-2e4803f64530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:52.126485 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da660f5-66d6-4d3f-a55c-2e4803f64530 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:52.142514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1206/2440] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:28:52.148272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ca31-e676-4b46-a3d0-abd2b8628abb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:52.158467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ca31-e676-4b46-a3d0-abd2b8628abb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:52.165020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:52.166306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:52.166685 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:52.167055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] PATH_INFO: `/v3/projects/838a10df12d44227accfaa07f43af24e/users/ec41bea1918141a28a18aab31aa59982/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:52.185132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=838a10df12d44227accfaa07f43af24e, user_id=ec41bea1918141a28a18aab31aa59982, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:52.187842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:52.202998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1235/2441] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:28:52 2022] PUT /identity/v3/projects/838a10df12d44227accfaa07f43af24e/users/ec41bea1918141a28a18aab31aa59982/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:28:52.213518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73df3d-5517-412b-822b-91421909c3bb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:52.213807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73df3d-5517-412b-822b-91421909c3bb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:52.214081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73df3d-5517-412b-822b-91421909c3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:52.241113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4e73df3d-5517-412b-822b-91421909c3bb None None] MFA Rules not processed for user `ec41bea1918141a28a18aab31aa59982`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:28:52.242660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4e73df3d-5517-412b-822b-91421909c3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:52.308927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1207/2442] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:28:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:28:52.323082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:28:52.326223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:28:52.386039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84caed-3d43-472e-a31a-005afb650d0a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:28:52.390044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb9dae-7087-47ad-bc75-809728bf887b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:28:52.390687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9dae-7087-47ad-bc75-809728bf887b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:28:52.390971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9dae-7087-47ad-bc75-809728bf887b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:28:52.391245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9dae-7087-47ad-bc75-809728bf887b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:28:52.400028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb9dae-7087-47ad-bc75-809728bf887b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:28:52.401484 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb9dae-7087-47ad-bc75-809728bf887b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:28:52.410947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1236/2443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:28:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:01.909044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73df3d-5517-412b-822b-91421909c3bb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:01.984122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73df3d-5517-412b-822b-91421909c3bb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:01.989664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:01.990545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:01.990826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:01.991470 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] PATH_INFO: `/v3/users/ec41bea1918141a28a18aab31aa59982` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:01.992704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec41bea1918141a28a18aab31aa59982)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:01.994638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:02.016110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:02.045054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec41bea1918141a28a18aab31aa59982'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.054049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec41bea1918141a28a18aab31aa59982'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.061176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec41bea1918141a28a18aab31aa59982'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.061813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec41bea1918141a28a18aab31aa59982'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.074336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec41bea1918141a28a18aab31aa59982'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.080418 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1208/2444] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:01 2022] DELETE /identity/v3/users/ec41bea1918141a28a18aab31aa59982 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:02.085411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea825b31-15cf-4604-a25a-a4f41131a200 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:02.085930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea825b31-15cf-4604-a25a-a4f41131a200 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:02.086045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea825b31-15cf-4604-a25a-a4f41131a200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:02.104339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ea825b31-15cf-4604-a25a-a4f41131a200 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:02.105545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ea825b31-15cf-4604-a25a-a4f41131a200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:02.168519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1237/2445] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:29:02.178579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:02.180227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:02.244078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd183787-4ee9-4514-b3c6-d262c2852316 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:02.248550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8e773-8d9d-4ff1-bc5a-279cf0ccbeae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:02.249195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8e773-8d9d-4ff1-bc5a-279cf0ccbeae service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:02.249480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8e773-8d9d-4ff1-bc5a-279cf0ccbeae service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:02.249739 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8e773-8d9d-4ff1-bc5a-279cf0ccbeae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:02.259965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8e773-8d9d-4ff1-bc5a-279cf0ccbeae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:02.261409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8e773-8d9d-4ff1-bc5a-279cf0ccbeae service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:02.271676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1209/2446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:02.562812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea825b31-15cf-4604-a25a-a4f41131a200 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:02.564582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ea825b31-15cf-4604-a25a-a4f41131a200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:02.579384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea825b31-15cf-4604-a25a-a4f41131a200 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:02.585401 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:02.586457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:02.586990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:02.587303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] PATH_INFO: `/v3/projects/838a10df12d44227accfaa07f43af24e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:02.588965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=838a10df12d44227accfaa07f43af24e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:02.591285 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:02.647572 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:02.649008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838a10df12d44227accfaa07f43af24e', 'request_id': 'req-43cf9b13-6f56-444b-b034-d9db6f0671a1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.656211 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838a10df12d44227accfaa07f43af24e', 'request_id': 'req-43cf9b13-6f56-444b-b034-d9db6f0671a1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.664655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838a10df12d44227accfaa07f43af24e', 'request_id': 'req-43cf9b13-6f56-444b-b034-d9db6f0671a1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.676287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] The token cache is being invalidate because project 838a10df12d44227accfaa07f43af24e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:02.676506 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.677143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43cf9b13-6f56-444b-b034-d9db6f0671a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:02.678729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1238/2447] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:02 2022] DELETE /identity/v3/projects/838a10df12d44227accfaa07f43af24e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:02.683760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f1583-69d2-45ff-8a86-2061453e6893 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:02.684052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f1583-69d2-45ff-8a86-2061453e6893 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:02.684329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f1583-69d2-45ff-8a86-2061453e6893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:02.703967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8d3f1583-69d2-45ff-8a86-2061453e6893 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:02.705175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8d3f1583-69d2-45ff-8a86-2061453e6893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:02.787522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1210/2448] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:29:02.798219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a95be-0b85-4a44-9008-10313857df78 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:02.798501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a95be-0b85-4a44-9008-10313857df78 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:02.798775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a95be-0b85-4a44-9008-10313857df78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:02.818843 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-674a95be-0b85-4a44-9008-10313857df78 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:02.820079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-674a95be-0b85-4a44-9008-10313857df78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:02.829043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1239/2449] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:02.834381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f1583-69d2-45ff-8a86-2061453e6893 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:02.850496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f1583-69d2-45ff-8a86-2061453e6893 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:02.857691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:02.858778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:02.859173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:02.859475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:02.860557 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:29:02.861035 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:02.862994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:02.876507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1211/2450] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:29:02.884543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a95be-0b85-4a44-9008-10313857df78 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:02.898083 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a95be-0b85-4a44-9008-10313857df78 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:02.904864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d3d24-0c6d-47fd-a23a-a29abe399c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:02.905536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d3d24-0c6d-47fd-a23a-a29abe399c10 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:02.905814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d3d24-0c6d-47fd-a23a-a29abe399c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:02.906078 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d3d24-0c6d-47fd-a23a-a29abe399c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:02.906777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d3d24-0c6d-47fd-a23a-a29abe399c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:02.908783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d3d24-0c6d-47fd-a23a-a29abe399c10 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:02.942015 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1240/2451] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:29:02 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:02.951578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:02.975331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a2358-9081-4661-8de4-a4cb290b55a7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:03.010462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:03.055635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519c2f4-efc9-4578-bfbb-3d89d5e36744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:03.056360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519c2f4-efc9-4578-bfbb-3d89d5e36744 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:03.056641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519c2f4-efc9-4578-bfbb-3d89d5e36744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:03.056907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519c2f4-efc9-4578-bfbb-3d89d5e36744 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:03.057649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519c2f4-efc9-4578-bfbb-3d89d5e36744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:03.059597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519c2f4-efc9-4578-bfbb-3d89d5e36744 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:03.088140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1212/2452] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:29:02 2022] POST /identity/v3/users => generated 302 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:03.092744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d3d24-0c6d-47fd-a23a-a29abe399c10 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:03.104454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d3d24-0c6d-47fd-a23a-a29abe399c10 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:03.109703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011af1bd-38c6-4a07-a630-71561bc5368f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:03.110382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011af1bd-38c6-4a07-a630-71561bc5368f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:03.110660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011af1bd-38c6-4a07-a630-71561bc5368f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:03.110930 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011af1bd-38c6-4a07-a630-71561bc5368f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:03.111704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011af1bd-38c6-4a07-a630-71561bc5368f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:03.113528 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011af1bd-38c6-4a07-a630-71561bc5368f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:03.126070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1241/2453] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:03.131482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519c2f4-efc9-4578-bfbb-3d89d5e36744 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:03.147286 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519c2f4-efc9-4578-bfbb-3d89d5e36744 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:03.154391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfdfb0-9791-456a-b875-683b8f359263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:03.155269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfdfb0-9791-456a-b875-683b8f359263 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:03.155718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfdfb0-9791-456a-b875-683b8f359263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:03.156070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfdfb0-9791-456a-b875-683b8f359263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:03.157071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfdfb0-9791-456a-b875-683b8f359263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:03.159911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfdfb0-9791-456a-b875-683b8f359263 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:03.170939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1213/2454] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:03.175512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011af1bd-38c6-4a07-a630-71561bc5368f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:03.186754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011af1bd-38c6-4a07-a630-71561bc5368f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:03.192744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:03.193564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:03.193842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:03.194115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] PATH_INFO: `/v3/projects/7167d99c29c14d17a15d7aaa45825a74/users/2280e660c2724e55b17441155bfa9a24/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:03.210865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7167d99c29c14d17a15d7aaa45825a74, user_id=2280e660c2724e55b17441155bfa9a24, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:03.213564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:03.234036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1242/2455] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:03 2022] PUT /identity/v3/projects/7167d99c29c14d17a15d7aaa45825a74/users/2280e660c2724e55b17441155bfa9a24/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:03.241449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0bd55-2609-4f70-b4d8-485de92698e4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:03.241740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0bd55-2609-4f70-b4d8-485de92698e4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:03.242039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0bd55-2609-4f70-b4d8-485de92698e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:03.264863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0fb0bd55-2609-4f70-b4d8-485de92698e4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:03.266102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0fb0bd55-2609-4f70-b4d8-485de92698e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:03.342145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1214/2456] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:03.356262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:03.358483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:03.432192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f388cb-a986-47dd-a8a7-8dd5aef29d6e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:03.437913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b930700-151d-42d5-9316-e22d528d8729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:03.438618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b930700-151d-42d5-9316-e22d528d8729 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:03.439070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b930700-151d-42d5-9316-e22d528d8729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:03.439706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b930700-151d-42d5-9316-e22d528d8729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:03.454573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b930700-151d-42d5-9316-e22d528d8729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:03.456010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b930700-151d-42d5-9316-e22d528d8729 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:03.469566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1243/2457] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:04.126794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0f47-0af8-4335-b4cd-bb08dd8397db None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:04.127362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0f47-0af8-4335-b4cd-bb08dd8397db None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:04.127362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0f47-0af8-4335-b4cd-bb08dd8397db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:04.155017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e0ef0f47-0af8-4335-b4cd-bb08dd8397db None None] MFA Rules not processed for user `2280e660c2724e55b17441155bfa9a24`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:04.156264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e0ef0f47-0af8-4335-b4cd-bb08dd8397db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:04.222405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1215/2458] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:04.227496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b930700-151d-42d5-9316-e22d528d8729 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:04.237333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b930700-151d-42d5-9316-e22d528d8729 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:04.242719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ade07e-bc8a-4ee3-b555-41045f8ed195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:04.243345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade07e-bc8a-4ee3-b555-41045f8ed195 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:04.243631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade07e-bc8a-4ee3-b555-41045f8ed195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:04.243905 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade07e-bc8a-4ee3-b555-41045f8ed195 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:04.244606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ade07e-bc8a-4ee3-b555-41045f8ed195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:04.246529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ade07e-bc8a-4ee3-b555-41045f8ed195 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:04.274577 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1244/2459] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:29:04 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:29:04.278806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef0f47-0af8-4335-b4cd-bb08dd8397db None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:04.290421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef0f47-0af8-4335-b4cd-bb08dd8397db None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:04.326343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:04.354943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e030c3a-c584-45f2-9d43-16a4179fa5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:04.355670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e030c3a-c584-45f2-9d43-16a4179fa5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:04.355951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e030c3a-c584-45f2-9d43-16a4179fa5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:04.356226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e030c3a-c584-45f2-9d43-16a4179fa5c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:04.356917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e030c3a-c584-45f2-9d43-16a4179fa5c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:04.358839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e030c3a-c584-45f2-9d43-16a4179fa5c2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:04.384987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1216/2460] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:29:04 2022] POST /identity/v3/users => generated 308 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:04.391327 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ade07e-bc8a-4ee3-b555-41045f8ed195 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:04.403511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ade07e-bc8a-4ee3-b555-41045f8ed195 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:04.408937 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213d6a9-d0d6-4fe8-8d01-774f3d1383a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:04.409653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213d6a9-d0d6-4fe8-8d01-774f3d1383a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:04.409936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213d6a9-d0d6-4fe8-8d01-774f3d1383a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:04.410216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213d6a9-d0d6-4fe8-8d01-774f3d1383a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:04.410947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213d6a9-d0d6-4fe8-8d01-774f3d1383a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:04.412615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213d6a9-d0d6-4fe8-8d01-774f3d1383a7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:04.423536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1245/2461] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:04.428018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e030c3a-c584-45f2-9d43-16a4179fa5c2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:04.438303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e030c3a-c584-45f2-9d43-16a4179fa5c2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:04.444441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46742c-c1da-4ee4-b093-52de276cf376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:04.445345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46742c-c1da-4ee4-b093-52de276cf376 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:04.445628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46742c-c1da-4ee4-b093-52de276cf376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:04.445954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46742c-c1da-4ee4-b093-52de276cf376 admin admin] PATH_INFO: `/v3/projects/cfd5d3bdd3824691ba0dea8971d7a909/users/2e810a2918e041a98719e4845d4b3875/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:04.461467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46742c-c1da-4ee4-b093-52de276cf376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd5d3bdd3824691ba0dea8971d7a909, user_id=2e810a2918e041a98719e4845d4b3875, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:04.464277 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46742c-c1da-4ee4-b093-52de276cf376 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:04.479192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1217/2462] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:04 2022] PUT /identity/v3/projects/cfd5d3bdd3824691ba0dea8971d7a909/users/2e810a2918e041a98719e4845d4b3875/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:29:04.483431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213d6a9-d0d6-4fe8-8d01-774f3d1383a7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:04.494298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213d6a9-d0d6-4fe8-8d01-774f3d1383a7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:04.538094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:04.557175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f67464-1251-4386-8a2c-710bc273126a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:04.557853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67464-1251-4386-8a2c-710bc273126a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:04.558136 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67464-1251-4386-8a2c-710bc273126a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:04.558436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67464-1251-4386-8a2c-710bc273126a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:04.559163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f67464-1251-4386-8a2c-710bc273126a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:04.560831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f67464-1251-4386-8a2c-710bc273126a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:04.570782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1246/2463] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:29:04.580578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46742c-c1da-4ee4-b093-52de276cf376 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:04.591698 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46742c-c1da-4ee4-b093-52de276cf376 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:04.596983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:04.597916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:04.598238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:04.598512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] PATH_INFO: `/v3/projects/cfd5d3bdd3824691ba0dea8971d7a909/users/2e810a2918e041a98719e4845d4b3875/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:04.600612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd5d3bdd3824691ba0dea8971d7a909, user_id=2e810a2918e041a98719e4845d4b3875, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:04.603310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:04.614075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1218/2464] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:04 2022] PUT /identity/v3/projects/cfd5d3bdd3824691ba0dea8971d7a909/users/2e810a2918e041a98719e4845d4b3875/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:05.244836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221a372-9d84-4165-8b4f-2397002e7987 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:05.245370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221a372-9d84-4165-8b4f-2397002e7987 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:05.245370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221a372-9d84-4165-8b4f-2397002e7987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:05.271671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d221a372-9d84-4165-8b4f-2397002e7987 None None] MFA Rules not processed for user `2e810a2918e041a98719e4845d4b3875`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:05.272885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d221a372-9d84-4165-8b4f-2397002e7987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:05.344954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1247/2465] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:05 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:06.319486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:06.321456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:06.380440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221a372-9d84-4165-8b4f-2397002e7987 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:06.403919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3da0db-cb1f-4ee7-be4e-358eeb8ebd50 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:06.409796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:06.410595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:06.410715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:06.410991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:06.412866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:06.424572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:06.426281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:06.436349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1219/2466] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:06.448043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221a372-9d84-4165-8b4f-2397002e7987 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:06.452925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:06.453822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:06.454106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:06.454382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] PATH_INFO: `/v3/users/2280e660c2724e55b17441155bfa9a24` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:06.455633 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2280e660c2724e55b17441155bfa9a24)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:06.458355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:06.511094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2280e660c2724e55b17441155bfa9a24'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.511959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2280e660c2724e55b17441155bfa9a24'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.520109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2280e660c2724e55b17441155bfa9a24'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.527522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:06.534069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2280e660c2724e55b17441155bfa9a24'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.543964 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2280e660c2724e55b17441155bfa9a24'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.549985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1248/2467] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:06 2022] DELETE /identity/v3/users/2280e660c2724e55b17441155bfa9a24 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:06.730667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:06.743317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bb243-9ec7-4bff-bbd6-f6b428255bcd service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:06.804956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:06.805755 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:06.806038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:06.806308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] PATH_INFO: `/v3/projects/7167d99c29c14d17a15d7aaa45825a74` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:06.807530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7167d99c29c14d17a15d7aaa45825a74)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:06.809482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:06.831909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:06.859138 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7167d99c29c14d17a15d7aaa45825a74', 'request_id': 'req-cc0b6f22-198a-4918-8124-7b4d661c044e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.864887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7167d99c29c14d17a15d7aaa45825a74', 'request_id': 'req-cc0b6f22-198a-4918-8124-7b4d661c044e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.875124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7167d99c29c14d17a15d7aaa45825a74', 'request_id': 'req-cc0b6f22-198a-4918-8124-7b4d661c044e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.889484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] The token cache is being invalidate because project 7167d99c29c14d17a15d7aaa45825a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:06.889830 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.890886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:06.893468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1220/2468] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:06 2022] DELETE /identity/v3/projects/7167d99c29c14d17a15d7aaa45825a74 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:06.899512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:06.901740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:06.918250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b57a6-bd05-46e1-97e3-9b0312c27653 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:06.983746 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:06.984642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:06.984902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:06.985201 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] PATH_INFO: `/v3/users/2e810a2918e041a98719e4845d4b3875` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:06.986553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e810a2918e041a98719e4845d4b3875)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:06.988863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:07.038204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e810a2918e041a98719e4845d4b3875'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.038843 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e810a2918e041a98719e4845d4b3875'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.045584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e810a2918e041a98719e4845d4b3875'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.058584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e810a2918e041a98719e4845d4b3875'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.067198 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d70cc3d2-431d-4c7e-bb50-810b3f42a823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e810a2918e041a98719e4845d4b3875'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.073508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1249/2469] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:06 2022] DELETE /identity/v3/users/2e810a2918e041a98719e4845d4b3875 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:07.254930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:07.256461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:07.271909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b6f22-198a-4918-8124-7b4d661c044e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:07.332693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:07.333530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.333818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.334104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] PATH_INFO: `/v3/projects/cfd5d3bdd3824691ba0dea8971d7a909` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.335421 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd5d3bdd3824691ba0dea8971d7a909)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:07.337375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:07.385431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd5d3bdd3824691ba0dea8971d7a909', 'request_id': 'req-0ca7780e-8016-4019-bc75-1d128b67732b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.390834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd5d3bdd3824691ba0dea8971d7a909', 'request_id': 'req-0ca7780e-8016-4019-bc75-1d128b67732b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.399235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd5d3bdd3824691ba0dea8971d7a909', 'request_id': 'req-0ca7780e-8016-4019-bc75-1d128b67732b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.410637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] The token cache is being invalidate because project cfd5d3bdd3824691ba0dea8971d7a909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:07.410937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.411617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0ca7780e-8016-4019-bc75-1d128b67732b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:07.413265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1221/2470] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:07 2022] DELETE /identity/v3/projects/cfd5d3bdd3824691ba0dea8971d7a909 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:07.418513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9ffd-8b58-4d3a-861f-b1276e11eeca None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.419044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9ffd-8b58-4d3a-861f-b1276e11eeca None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.419160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9ffd-8b58-4d3a-861f-b1276e11eeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.438152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a62f9ffd-8b58-4d3a-861f-b1276e11eeca None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:07.439400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a62f9ffd-8b58-4d3a-861f-b1276e11eeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:07.513184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1250/2471] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:07.523683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72d22-0dac-4404-a9fc-fbe71b431e27 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.524001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72d22-0dac-4404-a9fc-fbe71b431e27 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.524344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72d22-0dac-4404-a9fc-fbe71b431e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.548738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a5b72d22-0dac-4404-a9fc-fbe71b431e27 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:07.549959 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a5b72d22-0dac-4404-a9fc-fbe71b431e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:07.559633 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1222/2472] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:07.564157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f9ffd-8b58-4d3a-861f-b1276e11eeca None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:07.569244 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:07.575102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f9ffd-8b58-4d3a-861f-b1276e11eeca None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:07.581342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:07.582209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.582653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.583269 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.583949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:29:07.584259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:07.585932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:07.598076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1251/2473] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:29:07.602482 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72d22-0dac-4404-a9fc-fbe71b431e27 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:07.611928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72d22-0dac-4404-a9fc-fbe71b431e27 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:07.617384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d942a-2ae3-45a6-a268-10b946fb0caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:07.618036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d942a-2ae3-45a6-a268-10b946fb0caf admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.618319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d942a-2ae3-45a6-a268-10b946fb0caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.618592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d942a-2ae3-45a6-a268-10b946fb0caf admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.619324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d942a-2ae3-45a6-a268-10b946fb0caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:07.621225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d942a-2ae3-45a6-a268-10b946fb0caf admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:07.652845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1223/2474] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:29:07 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:07.657235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:07.668466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b84a1-bf10-41e5-b58d-6c901e998529 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:07.739449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c01b8-f772-4f1d-8420-d690528e9f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:07.740118 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c01b8-f772-4f1d-8420-d690528e9f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.740395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c01b8-f772-4f1d-8420-d690528e9f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.740667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c01b8-f772-4f1d-8420-d690528e9f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.741705 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c01b8-f772-4f1d-8420-d690528e9f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:07.743668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c01b8-f772-4f1d-8420-d690528e9f99 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:07.765019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1252/2475] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:29:07 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:07.771675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d942a-2ae3-45a6-a268-10b946fb0caf admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:07.787432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d942a-2ae3-45a6-a268-10b946fb0caf admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:07.793468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d760b07-4405-471d-b43b-bf6ecc6ce588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:07.794131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d760b07-4405-471d-b43b-bf6ecc6ce588 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.794409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d760b07-4405-471d-b43b-bf6ecc6ce588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.794694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d760b07-4405-471d-b43b-bf6ecc6ce588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.795428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d760b07-4405-471d-b43b-bf6ecc6ce588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:07.797030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d760b07-4405-471d-b43b-bf6ecc6ce588 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:07.802832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:07.808736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1224/2476] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:07.813348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c01b8-f772-4f1d-8420-d690528e9f99 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:07.823691 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c01b8-f772-4f1d-8420-d690528e9f99 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:07.828896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6add57-a3c3-4601-8e32-97cd9f97ed14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:07.829624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6add57-a3c3-4601-8e32-97cd9f97ed14 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.829923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6add57-a3c3-4601-8e32-97cd9f97ed14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.830215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6add57-a3c3-4601-8e32-97cd9f97ed14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.830962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6add57-a3c3-4601-8e32-97cd9f97ed14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:07.832842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6add57-a3c3-4601-8e32-97cd9f97ed14 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:07.844948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1253/2477] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:07.849182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d760b07-4405-471d-b43b-bf6ecc6ce588 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:07.859734 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d760b07-4405-471d-b43b-bf6ecc6ce588 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:07.865137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe335-d248-4341-a8c2-4fad7d93cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:07.866385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe335-d248-4341-a8c2-4fad7d93cac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.866663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe335-d248-4341-a8c2-4fad7d93cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.866933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe335-d248-4341-a8c2-4fad7d93cac5 admin admin] PATH_INFO: `/v3/projects/efd18ef7c4bd43cab5eeb2c1460cb0e6/users/2b0407b3dba54c06be51fd9f0e4a8da6/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.882221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fe335-d248-4341-a8c2-4fad7d93cac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd18ef7c4bd43cab5eeb2c1460cb0e6, user_id=2b0407b3dba54c06be51fd9f0e4a8da6, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:07.884980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fe335-d248-4341-a8c2-4fad7d93cac5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:07.901537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1225/2478] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:07 2022] PUT /identity/v3/projects/efd18ef7c4bd43cab5eeb2c1460cb0e6/users/2b0407b3dba54c06be51fd9f0e4a8da6/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:07.911952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e9325-a771-461e-b3c2-8620dc7ddbfd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:07.912242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e9325-a771-461e-b3c2-8620dc7ddbfd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:07.912519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e9325-a771-461e-b3c2-8620dc7ddbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:07.940001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cc0e9325-a771-461e-b3c2-8620dc7ddbfd None None] MFA Rules not processed for user `2b0407b3dba54c06be51fd9f0e4a8da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:07.941351 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cc0e9325-a771-461e-b3c2-8620dc7ddbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:08.009338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1254/2479] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:08.014101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe335-d248-4341-a8c2-4fad7d93cac5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:08.089406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe335-d248-4341-a8c2-4fad7d93cac5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:08.094819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b590270-baab-48e9-bb1e-589a9f0f73db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:08.095585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b590270-baab-48e9-bb1e-589a9f0f73db admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.095804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b590270-baab-48e9-bb1e-589a9f0f73db admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.096083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b590270-baab-48e9-bb1e-589a9f0f73db admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.096770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b590270-baab-48e9-bb1e-589a9f0f73db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:08.098746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b590270-baab-48e9-bb1e-589a9f0f73db admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:08.128974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1226/2480] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:29:08 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:08.133270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e9325-a771-461e-b3c2-8620dc7ddbfd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:08.145295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e9325-a771-461e-b3c2-8620dc7ddbfd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:08.199242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:08.207501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865104c-dc98-4eed-9ef3-91eb598afecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:08.208192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865104c-dc98-4eed-9ef3-91eb598afecf admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.208477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865104c-dc98-4eed-9ef3-91eb598afecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.208750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865104c-dc98-4eed-9ef3-91eb598afecf admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.209490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865104c-dc98-4eed-9ef3-91eb598afecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:08.211778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865104c-dc98-4eed-9ef3-91eb598afecf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:08.233906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1255/2481] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:29:08 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:29:08.238181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b590270-baab-48e9-bb1e-589a9f0f73db admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:08.249706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b590270-baab-48e9-bb1e-589a9f0f73db admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:08.255532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503ff62-aa70-47a4-9670-4b6a9e1e5eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:08.256192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503ff62-aa70-47a4-9670-4b6a9e1e5eea admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.256476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503ff62-aa70-47a4-9670-4b6a9e1e5eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.256747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503ff62-aa70-47a4-9670-4b6a9e1e5eea admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.257483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503ff62-aa70-47a4-9670-4b6a9e1e5eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:08.259120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503ff62-aa70-47a4-9670-4b6a9e1e5eea admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:08.270173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1227/2482] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:08.274381 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865104c-dc98-4eed-9ef3-91eb598afecf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:08.285430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865104c-dc98-4eed-9ef3-91eb598afecf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:08.291115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb382-f570-4469-89b6-5ef0f8f1d1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:08.292033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb382-f570-4469-89b6-5ef0f8f1d1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.292308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb382-f570-4469-89b6-5ef0f8f1d1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.292582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb382-f570-4469-89b6-5ef0f8f1d1f1 admin admin] PATH_INFO: `/v3/projects/9b1a00c6420e4e4a953694923e6442d3/users/caf9620f9b9c4eb7988207b956f651b4/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.309615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fb382-f570-4469-89b6-5ef0f8f1d1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1a00c6420e4e4a953694923e6442d3, user_id=caf9620f9b9c4eb7988207b956f651b4, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:08.312400 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fb382-f570-4469-89b6-5ef0f8f1d1f1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:08.327540 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1256/2483] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:08 2022] PUT /identity/v3/projects/9b1a00c6420e4e4a953694923e6442d3/users/caf9620f9b9c4eb7988207b956f651b4/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:08.331824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503ff62-aa70-47a4-9670-4b6a9e1e5eea admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:08.343167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503ff62-aa70-47a4-9670-4b6a9e1e5eea admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:08.411385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c01c5f-0a15-4f40-8eed-c32aa523c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:08.412057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c01c5f-0a15-4f40-8eed-c32aa523c228 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.412329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c01c5f-0a15-4f40-8eed-c32aa523c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.412605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c01c5f-0a15-4f40-8eed-c32aa523c228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.413340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c01c5f-0a15-4f40-8eed-c32aa523c228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:08.414977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c01c5f-0a15-4f40-8eed-c32aa523c228 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:08.425356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1228/2484] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:08.431322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb382-f570-4469-89b6-5ef0f8f1d1f1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:08.442576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb382-f570-4469-89b6-5ef0f8f1d1f1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:08.447971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:08.448851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.449129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.449398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] PATH_INFO: `/v3/projects/9b1a00c6420e4e4a953694923e6442d3/users/caf9620f9b9c4eb7988207b956f651b4/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.451663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1a00c6420e4e4a953694923e6442d3, user_id=caf9620f9b9c4eb7988207b956f651b4, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:08.454315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:08.465202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1257/2485] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:08 2022] PUT /identity/v3/projects/9b1a00c6420e4e4a953694923e6442d3/users/caf9620f9b9c4eb7988207b956f651b4/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:08.479510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5597a07-0ce6-47d7-a128-82b97bceefe8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.479804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5597a07-0ce6-47d7-a128-82b97bceefe8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.480081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5597a07-0ce6-47d7-a128-82b97bceefe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.514605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d5597a07-0ce6-47d7-a128-82b97bceefe8 None None] MFA Rules not processed for user `caf9620f9b9c4eb7988207b956f651b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:08.515866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d5597a07-0ce6-47d7-a128-82b97bceefe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:08.553798 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:08.592891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1229/2486] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:08.603990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:08.606064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:08.667028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867260a-7258-475b-b7e1-0051dffb34c0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:08.671102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae880781-23c8-486e-8880-c343a3679ab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:08.671741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae880781-23c8-486e-8880-c343a3679ab3 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.672062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae880781-23c8-486e-8880-c343a3679ab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.672333 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae880781-23c8-486e-8880-c343a3679ab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.682181 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae880781-23c8-486e-8880-c343a3679ab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:08.683675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae880781-23c8-486e-8880-c343a3679ab3 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:08.693310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1258/2487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:08.739392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5597a07-0ce6-47d7-a128-82b97bceefe8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:08.741213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d5597a07-0ce6-47d7-a128-82b97bceefe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:08.753724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5597a07-0ce6-47d7-a128-82b97bceefe8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:08.757543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d9228-8f04-46c8-8d5c-a3bdddb5a612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:08.758201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d9228-8f04-46c8-8d5c-a3bdddb5a612 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:08.758486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d9228-8f04-46c8-8d5c-a3bdddb5a612 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:08.758758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d9228-8f04-46c8-8d5c-a3bdddb5a612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:08.819475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d9228-8f04-46c8-8d5c-a3bdddb5a612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:08.820928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d9228-8f04-46c8-8d5c-a3bdddb5a612 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:08.830088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1230/2488] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:29:13.318706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae880781-23c8-486e-8880-c343a3679ab3 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:13.391092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae880781-23c8-486e-8880-c343a3679ab3 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:13.396386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:13.397264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:13.397542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:13.397816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] PATH_INFO: `/v3/users/2b0407b3dba54c06be51fd9f0e4a8da6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:13.399038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b0407b3dba54c06be51fd9f0e4a8da6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:13.401030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:13.454528 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b0407b3dba54c06be51fd9f0e4a8da6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:13.455174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b0407b3dba54c06be51fd9f0e4a8da6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:13.460980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:13.463337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b0407b3dba54c06be51fd9f0e4a8da6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:13.476955 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b0407b3dba54c06be51fd9f0e4a8da6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:13.486202 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b0407b3dba54c06be51fd9f0e4a8da6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:13.495010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1259/2489] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:13 2022] DELETE /identity/v3/users/2b0407b3dba54c06be51fd9f0e4a8da6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:13.499870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28194345-1163-4dde-8632-e4b574db9d2f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:13.500384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28194345-1163-4dde-8632-e4b574db9d2f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:13.500486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28194345-1163-4dde-8632-e4b574db9d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:13.519676 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-28194345-1163-4dde-8632-e4b574db9d2f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:13.521017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-28194345-1163-4dde-8632-e4b574db9d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:13.580409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1231/2490] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:13.591742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:13.593399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:13.653476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435fe2c-a786-4f40-a54c-7a559409173f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:13.657271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57419714-7a88-4145-9d3c-589acd943289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:13.657923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57419714-7a88-4145-9d3c-589acd943289 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:13.658203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57419714-7a88-4145-9d3c-589acd943289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:13.658477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57419714-7a88-4145-9d3c-589acd943289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:13.668389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57419714-7a88-4145-9d3c-589acd943289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:13.669956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57419714-7a88-4145-9d3c-589acd943289 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:13.680067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1260/2491] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:29:13.960063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28194345-1163-4dde-8632-e4b574db9d2f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:13.961746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-28194345-1163-4dde-8632-e4b574db9d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:13.974723 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28194345-1163-4dde-8632-e4b574db9d2f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:13.980125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:13.981033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:13.981304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:13.981709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] PATH_INFO: `/v3/projects/efd18ef7c4bd43cab5eeb2c1460cb0e6` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:13.983752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd18ef7c4bd43cab5eeb2c1460cb0e6)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:13.985759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:14.042777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd18ef7c4bd43cab5eeb2c1460cb0e6', 'request_id': 'req-bb25598f-ebd9-49a6-87cb-1e2623f66684'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.049724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd18ef7c4bd43cab5eeb2c1460cb0e6', 'request_id': 'req-bb25598f-ebd9-49a6-87cb-1e2623f66684'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.058848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd18ef7c4bd43cab5eeb2c1460cb0e6', 'request_id': 'req-bb25598f-ebd9-49a6-87cb-1e2623f66684'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.060015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:14.073888 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] The token cache is being invalidate because project efd18ef7c4bd43cab5eeb2c1460cb0e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:14.074182 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.074982 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.076614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1232/2492] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:13 2022] DELETE /identity/v3/projects/efd18ef7c4bd43cab5eeb2c1460cb0e6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:14.081088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57419714-7a88-4145-9d3c-589acd943289 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:14.093353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57419714-7a88-4145-9d3c-589acd943289 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:14.150734 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:14.151984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:14.152461 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:14.152900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] PATH_INFO: `/v3/users/caf9620f9b9c4eb7988207b956f651b4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:14.154384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf9620f9b9c4eb7988207b956f651b4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:14.156638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:14.205670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf9620f9b9c4eb7988207b956f651b4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.206272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf9620f9b9c4eb7988207b956f651b4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.213641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf9620f9b9c4eb7988207b956f651b4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.226982 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf9620f9b9c4eb7988207b956f651b4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.229648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:14.236223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10879303-c691-451b-94b4-07be46169a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf9620f9b9c4eb7988207b956f651b4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.243949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1261/2493] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:14 2022] DELETE /identity/v3/users/caf9620f9b9c4eb7988207b956f651b4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:29:14.521995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:14.523541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:14.538457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25598f-ebd9-49a6-87cb-1e2623f66684 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:14.624387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:14.625270 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:14.625550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:14.625825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] PATH_INFO: `/v3/projects/9b1a00c6420e4e4a953694923e6442d3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:14.627068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b1a00c6420e4e4a953694923e6442d3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:14.629198 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:14.680033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b1a00c6420e4e4a953694923e6442d3', 'request_id': 'req-b592dc4b-4a21-4a84-a12a-73587f96d68f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.685640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b1a00c6420e4e4a953694923e6442d3', 'request_id': 'req-b592dc4b-4a21-4a84-a12a-73587f96d68f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.695345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b1a00c6420e4e4a953694923e6442d3', 'request_id': 'req-b592dc4b-4a21-4a84-a12a-73587f96d68f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.706824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] The token cache is being invalidate because project 9b1a00c6420e4e4a953694923e6442d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:14.707132 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.707842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b592dc4b-4a21-4a84-a12a-73587f96d68f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:14.709424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1233/2494] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:14 2022] DELETE /identity/v3/projects/9b1a00c6420e4e4a953694923e6442d3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:14.714615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dcc8d-f433-4195-82f2-0b9f45f4417e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:14.715071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dcc8d-f433-4195-82f2-0b9f45f4417e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:14.715321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dcc8d-f433-4195-82f2-0b9f45f4417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:14.732997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-933dcc8d-f433-4195-82f2-0b9f45f4417e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:14.734220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-933dcc8d-f433-4195-82f2-0b9f45f4417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:14.795114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1262/2495] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:14.803618 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cb218-95d9-4daa-ad57-c530d057247b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:14.803736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cb218-95d9-4daa-ad57-c530d057247b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:14.804027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cb218-95d9-4daa-ad57-c530d057247b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:14.823353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1c7cb218-95d9-4daa-ad57-c530d057247b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:14.824570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1c7cb218-95d9-4daa-ad57-c530d057247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:14.835071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1234/2496] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:14.839383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dcc8d-f433-4195-82f2-0b9f45f4417e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:14.849942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dcc8d-f433-4195-82f2-0b9f45f4417e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:14.854710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:14.855411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:14.855689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:14.855962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:14.856687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:29:14.857011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:14.858656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:14.873173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1263/2497] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:29:14.878574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cb218-95d9-4daa-ad57-c530d057247b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:14.888580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cb218-95d9-4daa-ad57-c530d057247b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:14.894226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52025e-5d1a-47f5-9c3a-9a37ff42d05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:14.895354 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52025e-5d1a-47f5-9c3a-9a37ff42d05c admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:14.895807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52025e-5d1a-47f5-9c3a-9a37ff42d05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:14.896254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52025e-5d1a-47f5-9c3a-9a37ff42d05c admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:14.897450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52025e-5d1a-47f5-9c3a-9a37ff42d05c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:14.900597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52025e-5d1a-47f5-9c3a-9a37ff42d05c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:14.941610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1235/2498] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:29:14 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:14.945776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:14.956854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f85bd-ee37-4e0d-b27d-a45e8e939d7c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.018846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe675a7-726c-4990-826e-dd703e274dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.019576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe675a7-726c-4990-826e-dd703e274dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.020095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe675a7-726c-4990-826e-dd703e274dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.020095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe675a7-726c-4990-826e-dd703e274dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.021085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe675a7-726c-4990-826e-dd703e274dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.023054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe675a7-726c-4990-826e-dd703e274dc9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.051173 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1264/2499] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:29:14 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:15.055724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52025e-5d1a-47f5-9c3a-9a37ff42d05c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.067405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52025e-5d1a-47f5-9c3a-9a37ff42d05c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.073464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c570a-a122-4352-bb21-6df8f07cafde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.074131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c570a-a122-4352-bb21-6df8f07cafde admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.074410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c570a-a122-4352-bb21-6df8f07cafde admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.074680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c570a-a122-4352-bb21-6df8f07cafde admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.075450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c570a-a122-4352-bb21-6df8f07cafde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.077060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c570a-a122-4352-bb21-6df8f07cafde admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.088053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1236/2500] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:15.092231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe675a7-726c-4990-826e-dd703e274dc9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.102903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe675a7-726c-4990-826e-dd703e274dc9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.108870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500aeaf-bed7-43ee-b4b6-30de11cd35d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.109532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500aeaf-bed7-43ee-b4b6-30de11cd35d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.109806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500aeaf-bed7-43ee-b4b6-30de11cd35d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.110074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500aeaf-bed7-43ee-b4b6-30de11cd35d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.110791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500aeaf-bed7-43ee-b4b6-30de11cd35d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.112431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500aeaf-bed7-43ee-b4b6-30de11cd35d8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.123479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1265/2501] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:15.127787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c570a-a122-4352-bb21-6df8f07cafde admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.137986 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c570a-a122-4352-bb21-6df8f07cafde admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.142867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0497c-533c-4d0f-b3a0-ddf90bd0dcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.143783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0497c-533c-4d0f-b3a0-ddf90bd0dcce admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.144059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0497c-533c-4d0f-b3a0-ddf90bd0dcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.144328 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0497c-533c-4d0f-b3a0-ddf90bd0dcce admin admin] PATH_INFO: `/v3/projects/0aab45f27eae4f408595e81a8cb2951b/users/c9c6ad899a4f4bd5bbf1003e7e24fcca/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.159340 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0497c-533c-4d0f-b3a0-ddf90bd0dcce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aab45f27eae4f408595e81a8cb2951b, user_id=c9c6ad899a4f4bd5bbf1003e7e24fcca, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.162052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0497c-533c-4d0f-b3a0-ddf90bd0dcce admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.177617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1237/2502] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:15 2022] PUT /identity/v3/projects/0aab45f27eae4f408595e81a8cb2951b/users/c9c6ad899a4f4bd5bbf1003e7e24fcca/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:15.188331 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde169-45d0-4052-956f-1ff0a443df81 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.188631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde169-45d0-4052-956f-1ff0a443df81 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.188912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde169-45d0-4052-956f-1ff0a443df81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.223155 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bbdde169-45d0-4052-956f-1ff0a443df81 None None] MFA Rules not processed for user `c9c6ad899a4f4bd5bbf1003e7e24fcca`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:15.224824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bbdde169-45d0-4052-956f-1ff0a443df81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:15.285164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:15.305353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1266/2503] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:15.310030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0497c-533c-4d0f-b3a0-ddf90bd0dcce admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.332288 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:15.379621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0497c-533c-4d0f-b3a0-ddf90bd0dcce admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.384899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901f48-5f26-4b41-9026-b128ac2491c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.385572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901f48-5f26-4b41-9026-b128ac2491c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.385856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901f48-5f26-4b41-9026-b128ac2491c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.386129 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901f48-5f26-4b41-9026-b128ac2491c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.386840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3901f48-5f26-4b41-9026-b128ac2491c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.388824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3901f48-5f26-4b41-9026-b128ac2491c6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.417465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1238/2504] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:29:15 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:15.421770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdde169-45d0-4052-956f-1ff0a443df81 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.432970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdde169-45d0-4052-956f-1ff0a443df81 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.491722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fcd68-c2a4-4f45-ae02-ba230b8a60e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.492406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fcd68-c2a4-4f45-ae02-ba230b8a60e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.492688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fcd68-c2a4-4f45-ae02-ba230b8a60e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.492966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fcd68-c2a4-4f45-ae02-ba230b8a60e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.493665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fcd68-c2a4-4f45-ae02-ba230b8a60e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.495652 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fcd68-c2a4-4f45-ae02-ba230b8a60e8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.518428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1267/2505] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:29:15 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:29:15.522590 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901f48-5f26-4b41-9026-b128ac2491c6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.537814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901f48-5f26-4b41-9026-b128ac2491c6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.542706 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53fde1-b0f2-4d5f-8617-6f3a04fd5358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.543407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53fde1-b0f2-4d5f-8617-6f3a04fd5358 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.543689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53fde1-b0f2-4d5f-8617-6f3a04fd5358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.543966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53fde1-b0f2-4d5f-8617-6f3a04fd5358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.544700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53fde1-b0f2-4d5f-8617-6f3a04fd5358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.546324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53fde1-b0f2-4d5f-8617-6f3a04fd5358 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.557561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1239/2506] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:15.562071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fcd68-c2a4-4f45-ae02-ba230b8a60e8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.572507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fcd68-c2a4-4f45-ae02-ba230b8a60e8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.577660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873578b-da19-4926-b78e-8dd522fd6a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.578549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873578b-da19-4926-b78e-8dd522fd6a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.578827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873578b-da19-4926-b78e-8dd522fd6a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.579097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873578b-da19-4926-b78e-8dd522fd6a23 admin admin] PATH_INFO: `/v3/projects/76dc6d19d2c344e8a4d0fa9c91ff7d1a/users/5695fa1c423f41f1b7b516e112762006/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.594383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873578b-da19-4926-b78e-8dd522fd6a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76dc6d19d2c344e8a4d0fa9c91ff7d1a, user_id=5695fa1c423f41f1b7b516e112762006, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.597146 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873578b-da19-4926-b78e-8dd522fd6a23 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.617969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1268/2507] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:15 2022] PUT /identity/v3/projects/76dc6d19d2c344e8a4d0fa9c91ff7d1a/users/5695fa1c423f41f1b7b516e112762006/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:15.622914 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53fde1-b0f2-4d5f-8617-6f3a04fd5358 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.636944 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53fde1-b0f2-4d5f-8617-6f3a04fd5358 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.700353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc4b1c-c41b-4e7f-815a-3c02180c9d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.701017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc4b1c-c41b-4e7f-815a-3c02180c9d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.701294 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc4b1c-c41b-4e7f-815a-3c02180c9d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.701569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc4b1c-c41b-4e7f-815a-3c02180c9d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.702292 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc4b1c-c41b-4e7f-815a-3c02180c9d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.704015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc4b1c-c41b-4e7f-815a-3c02180c9d49 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.714428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1240/2508] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:15.718859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873578b-da19-4926-b78e-8dd522fd6a23 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.730303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873578b-da19-4926-b78e-8dd522fd6a23 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.735969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.736871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.737145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.737437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] PATH_INFO: `/v3/projects/76dc6d19d2c344e8a4d0fa9c91ff7d1a/users/5695fa1c423f41f1b7b516e112762006/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.739592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=76dc6d19d2c344e8a4d0fa9c91ff7d1a, user_id=5695fa1c423f41f1b7b516e112762006, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.742562 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.754185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1269/2509] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:15 2022] PUT /identity/v3/projects/76dc6d19d2c344e8a4d0fa9c91ff7d1a/users/5695fa1c423f41f1b7b516e112762006/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:15.765909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3324fe-3082-4cc1-b343-ed8a8efdd6ad None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.766282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3324fe-3082-4cc1-b343-ed8a8efdd6ad None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.766463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3324fe-3082-4cc1-b343-ed8a8efdd6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.793794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bf3324fe-3082-4cc1-b343-ed8a8efdd6ad None None] MFA Rules not processed for user `5695fa1c423f41f1b7b516e112762006`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:15.794942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bf3324fe-3082-4cc1-b343-ed8a8efdd6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:15.873074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1241/2510] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:15.885055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:15.886914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:15.947903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a932c-83ac-4407-ae95-33dd1909e87e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:15.951673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82922cb6-ca24-49ba-b33d-9764589514b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:15.952314 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922cb6-ca24-49ba-b33d-9764589514b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:15.952593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922cb6-ca24-49ba-b33d-9764589514b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:15.952864 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922cb6-ca24-49ba-b33d-9764589514b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:15.961958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82922cb6-ca24-49ba-b33d-9764589514b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:15.963431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82922cb6-ca24-49ba-b33d-9764589514b7 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:15.972592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1270/2511] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:16.061902 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3324fe-3082-4cc1-b343-ed8a8efdd6ad None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:16.072144 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3324fe-3082-4cc1-b343-ed8a8efdd6ad None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:16.076389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e209da5-b9c6-42ca-abc5-b4b19d36a6e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:16.077036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209da5-b9c6-42ca-abc5-b4b19d36a6e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:16.077315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209da5-b9c6-42ca-abc5-b4b19d36a6e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:16.077582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209da5-b9c6-42ca-abc5-b4b19d36a6e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:16.091264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:16.148781 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e209da5-b9c6-42ca-abc5-b4b19d36a6e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:16.150860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e209da5-b9c6-42ca-abc5-b4b19d36a6e6 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:16.164728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1242/2512] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:21.019741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82922cb6-ca24-49ba-b33d-9764589514b7 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:21.069357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:21.096428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82922cb6-ca24-49ba-b33d-9764589514b7 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:21.101089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:21.101936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:21.102215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:21.102486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] PATH_INFO: `/v3/users/c9c6ad899a4f4bd5bbf1003e7e24fcca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:21.103716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c6ad899a4f4bd5bbf1003e7e24fcca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:21.105661 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:21.160599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c6ad899a4f4bd5bbf1003e7e24fcca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.161266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c6ad899a4f4bd5bbf1003e7e24fcca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.168206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c6ad899a4f4bd5bbf1003e7e24fcca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.180440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c6ad899a4f4bd5bbf1003e7e24fcca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.189252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c6ad899a4f4bd5bbf1003e7e24fcca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.195363 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1271/2513] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:21 2022] DELETE /identity/v3/users/c9c6ad899a4f4bd5bbf1003e7e24fcca => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:21.200060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b549-50d7-4812-a649-3a81ba7ba8b2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:21.200534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b549-50d7-4812-a649-3a81ba7ba8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:21.200660 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b549-50d7-4812-a649-3a81ba7ba8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:21.218854 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cd41b549-50d7-4812-a649-3a81ba7ba8b2 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:21.220120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cd41b549-50d7-4812-a649-3a81ba7ba8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:21.283282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1243/2514] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:21.293848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:21.295483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:21.357192 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18f55-aa22-4d17-95be-846bd792171b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:21.360863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ec1a0-e415-439d-a6dc-7fbf77f8482c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:21.361664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ec1a0-e415-439d-a6dc-7fbf77f8482c service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:21.362055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ec1a0-e415-439d-a6dc-7fbf77f8482c service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:21.362492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ec1a0-e415-439d-a6dc-7fbf77f8482c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:21.372207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ec1a0-e415-439d-a6dc-7fbf77f8482c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:21.373668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ec1a0-e415-439d-a6dc-7fbf77f8482c service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:21.383180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1272/2515] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:21.746914 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41b549-50d7-4812-a649-3a81ba7ba8b2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:21.748388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cd41b549-50d7-4812-a649-3a81ba7ba8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:21.761793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41b549-50d7-4812-a649-3a81ba7ba8b2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:21.766658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:21.767681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:21.767966 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:21.768283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] PATH_INFO: `/v3/projects/0aab45f27eae4f408595e81a8cb2951b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:21.769546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aab45f27eae4f408595e81a8cb2951b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:21.771529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:21.824266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aab45f27eae4f408595e81a8cb2951b', 'request_id': 'req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.830840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aab45f27eae4f408595e81a8cb2951b', 'request_id': 'req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.841459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aab45f27eae4f408595e81a8cb2951b', 'request_id': 'req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.853843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] The token cache is being invalidate because project 0aab45f27eae4f408595e81a8cb2951b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:21.854123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.854819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.856595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1244/2516] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:21 2022] DELETE /identity/v3/projects/0aab45f27eae4f408595e81a8cb2951b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:21.862721 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ec1a0-e415-439d-a6dc-7fbf77f8482c service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:21.875184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ec1a0-e415-439d-a6dc-7fbf77f8482c service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:21.933928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:21.934838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:21.935114 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:21.935485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] PATH_INFO: `/v3/users/5695fa1c423f41f1b7b516e112762006` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:21.936711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5695fa1c423f41f1b7b516e112762006)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:21.938648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:21.943976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:21.989091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5695fa1c423f41f1b7b516e112762006'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.989750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5695fa1c423f41f1b7b516e112762006'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:21.997119 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5695fa1c423f41f1b7b516e112762006'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:22.008801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5695fa1c423f41f1b7b516e112762006'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:22.017553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3aa6edd3-2024-4eae-a60e-bbd4812582c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5695fa1c423f41f1b7b516e112762006'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:22.024282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1273/2517] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:21 2022] DELETE /identity/v3/users/5695fa1c423f41f1b7b516e112762006 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:22.298011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:22.299513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:22.314771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c72fd9-0c7d-414a-9b9f-b99d7a1ff4ca admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:22.352801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:22.377717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:22.378522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.378835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.379111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] PATH_INFO: `/v3/projects/76dc6d19d2c344e8a4d0fa9c91ff7d1a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.380337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76dc6d19d2c344e8a4d0fa9c91ff7d1a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:22.382289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:22.431763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76dc6d19d2c344e8a4d0fa9c91ff7d1a', 'request_id': 'req-02b6d2d8-cd14-402d-8206-c87eac1028c4'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:22.437282 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76dc6d19d2c344e8a4d0fa9c91ff7d1a', 'request_id': 'req-02b6d2d8-cd14-402d-8206-c87eac1028c4'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:22.446059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76dc6d19d2c344e8a4d0fa9c91ff7d1a', 'request_id': 'req-02b6d2d8-cd14-402d-8206-c87eac1028c4'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:22.457326 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] The token cache is being invalidate because project 76dc6d19d2c344e8a4d0fa9c91ff7d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:22.457615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:22.458336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02b6d2d8-cd14-402d-8206-c87eac1028c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:22.460147 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1245/2518] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:22 2022] DELETE /identity/v3/projects/76dc6d19d2c344e8a4d0fa9c91ff7d1a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:22.465167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e42b0-10ff-4224-8a8f-3f57b6046195 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.465455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e42b0-10ff-4224-8a8f-3f57b6046195 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.465767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e42b0-10ff-4224-8a8f-3f57b6046195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.483913 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a08e42b0-10ff-4224-8a8f-3f57b6046195 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:22.485109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a08e42b0-10ff-4224-8a8f-3f57b6046195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:22.547332 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1274/2519] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:22.555680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437ae6-1517-44b1-b062-c0aa8818d332 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.555973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437ae6-1517-44b1-b062-c0aa8818d332 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.556253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437ae6-1517-44b1-b062-c0aa8818d332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.583009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ea437ae6-1517-44b1-b062-c0aa8818d332 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:22.584246 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ea437ae6-1517-44b1-b062-c0aa8818d332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:22.593976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1246/2520] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:22.598344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e42b0-10ff-4224-8a8f-3f57b6046195 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:22.608809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e42b0-10ff-4224-8a8f-3f57b6046195 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:22.614344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:22.615061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.615374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.615978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.616801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:29:22.617107 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:22.618736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:22.630563 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1275/2521] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:29:22.635083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea437ae6-1517-44b1-b062-c0aa8818d332 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:22.644495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea437ae6-1517-44b1-b062-c0aa8818d332 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:22.649388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d279e71-ecca-4923-93f7-a93f2dc4054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:22.650041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d279e71-ecca-4923-93f7-a93f2dc4054f admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.650321 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d279e71-ecca-4923-93f7-a93f2dc4054f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.650614 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d279e71-ecca-4923-93f7-a93f2dc4054f admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.651291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d279e71-ecca-4923-93f7-a93f2dc4054f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:22.653206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d279e71-ecca-4923-93f7-a93f2dc4054f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:22.680150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1247/2522] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:29:22 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:22.684424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:22.694411 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fca608-ed65-45d1-9c23-2d99b7febdc5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:22.752623 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b19df-b0b1-4998-a029-b644e8a3807c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:22.753300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b19df-b0b1-4998-a029-b644e8a3807c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.753584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b19df-b0b1-4998-a029-b644e8a3807c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.753859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b19df-b0b1-4998-a029-b644e8a3807c admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.754558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b19df-b0b1-4998-a029-b644e8a3807c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:22.756663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b19df-b0b1-4998-a029-b644e8a3807c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:22.778313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1276/2523] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:29:22 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:29:22.782838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d279e71-ecca-4923-93f7-a93f2dc4054f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:22.793670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d279e71-ecca-4923-93f7-a93f2dc4054f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:22.798531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c75c8e-aa95-4168-962f-720c2b7330b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:22.799223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c75c8e-aa95-4168-962f-720c2b7330b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.799519 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c75c8e-aa95-4168-962f-720c2b7330b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.799792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c75c8e-aa95-4168-962f-720c2b7330b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.800523 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c75c8e-aa95-4168-962f-720c2b7330b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:22.802167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c75c8e-aa95-4168-962f-720c2b7330b2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:22.813214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1248/2524] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:22.817389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b19df-b0b1-4998-a029-b644e8a3807c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:22.822755 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:22.828575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b19df-b0b1-4998-a029-b644e8a3807c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:22.834442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dd39d-af3e-435a-a6f0-e1c8dfb39145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:22.835828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd39d-af3e-435a-a6f0-e1c8dfb39145 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.835828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd39d-af3e-435a-a6f0-e1c8dfb39145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.836064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd39d-af3e-435a-a6f0-e1c8dfb39145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.837054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dd39d-af3e-435a-a6f0-e1c8dfb39145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:22.838940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dd39d-af3e-435a-a6f0-e1c8dfb39145 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:22.850391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1277/2525] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:29:22.854788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c75c8e-aa95-4168-962f-720c2b7330b2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:22.869015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c75c8e-aa95-4168-962f-720c2b7330b2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:22.874450 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:22.875422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.875703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.875990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] PATH_INFO: `/v3/projects/ff1b9750869d4b0084bf40bf99c7abb3/users/b1acf027c846465698f0b83c24685a58/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.892238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1b9750869d4b0084bf40bf99c7abb3, user_id=b1acf027c846465698f0b83c24685a58, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:22.895085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:22.911857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1249/2526] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:22 2022] PUT /identity/v3/projects/ff1b9750869d4b0084bf40bf99c7abb3/users/b1acf027c846465698f0b83c24685a58/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:22.922375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40f8d-ef18-4a91-af67-9964da359665 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:22.922663 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40f8d-ef18-4a91-af67-9964da359665 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:22.922994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40f8d-ef18-4a91-af67-9964da359665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:22.949236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-12f40f8d-ef18-4a91-af67-9964da359665 None None] MFA Rules not processed for user `b1acf027c846465698f0b83c24685a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:22.950453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-12f40f8d-ef18-4a91-af67-9964da359665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:23.014328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1278/2527] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:23.025639 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:23.027491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:23.086025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333e98b-f408-4b0c-b19e-7df6e7a8e852 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:23.090351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09654875-b2f0-41ca-b04c-ce73c2c54130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:23.091001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09654875-b2f0-41ca-b04c-ce73c2c54130 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:23.091402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09654875-b2f0-41ca-b04c-ce73c2c54130 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:23.091703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09654875-b2f0-41ca-b04c-ce73c2c54130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:23.101857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09654875-b2f0-41ca-b04c-ce73c2c54130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:23.103488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09654875-b2f0-41ca-b04c-ce73c2c54130 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:23.114454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1250/2528] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:23.133591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40f8d-ef18-4a91-af67-9964da359665 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:23.201271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40f8d-ef18-4a91-af67-9964da359665 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:23.206285 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:23.207196 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:23.207502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:23.207774 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] PATH_INFO: `/v3/users/b1acf027c846465698f0b83c24685a58` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:23.209028 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1acf027c846465698f0b83c24685a58)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:23.211352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:23.265880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1acf027c846465698f0b83c24685a58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:23.266652 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1acf027c846465698f0b83c24685a58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:23.274684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1acf027c846465698f0b83c24685a58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:23.288854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1acf027c846465698f0b83c24685a58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:23.300020 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1acf027c846465698f0b83c24685a58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:23.302603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:23.311612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1279/2529] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:23 2022] DELETE /identity/v3/users/b1acf027c846465698f0b83c24685a58 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:23.313528 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eecf-c81b-4120-9f18-ece69a60c6e4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:23.314428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eecf-c81b-4120-9f18-ece69a60c6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:23.314896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eecf-c81b-4120-9f18-ece69a60c6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:23.340802 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:23.356578 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-aed9eecf-c81b-4120-9f18-ece69a60c6e4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:23.358121 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-aed9eecf-c81b-4120-9f18-ece69a60c6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:23.436513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1251/2530] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:23.447054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:23.448884 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:23.522618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982f7ce-0a9a-434f-83aa-de2011067c27 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:23.526900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b946f-34f0-4a37-995c-4aca7c2165b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:23.527831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b946f-34f0-4a37-995c-4aca7c2165b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:23.528309 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b946f-34f0-4a37-995c-4aca7c2165b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:23.528585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b946f-34f0-4a37-995c-4aca7c2165b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:23.538991 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b946f-34f0-4a37-995c-4aca7c2165b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:23.540608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b946f-34f0-4a37-995c-4aca7c2165b2 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:23.550610 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1280/2531] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:24.445362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9eecf-c81b-4120-9f18-ece69a60c6e4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:24.447014 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-aed9eecf-c81b-4120-9f18-ece69a60c6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:24.460381 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9eecf-c81b-4120-9f18-ece69a60c6e4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:24.465998 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:24.467448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:24.467448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:24.467733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] PATH_INFO: `/v3/projects/ff1b9750869d4b0084bf40bf99c7abb3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:24.469006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1b9750869d4b0084bf40bf99c7abb3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:24.471032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:24.556845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1b9750869d4b0084bf40bf99c7abb3', 'request_id': 'req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:24.564092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1b9750869d4b0084bf40bf99c7abb3', 'request_id': 'req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:24.575931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1b9750869d4b0084bf40bf99c7abb3', 'request_id': 'req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:24.593544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] The token cache is being invalidate because project ff1b9750869d4b0084bf40bf99c7abb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:24.593951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:24.594972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a35cc56e-7fda-4b2f-8e71-60152cf37e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:24.600658 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1252/2532] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:24 2022] DELETE /identity/v3/projects/ff1b9750869d4b0084bf40bf99c7abb3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:24.602429 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943cbc3-334a-44c9-85f5-e23c7cff2560 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:24.602987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943cbc3-334a-44c9-85f5-e23c7cff2560 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:24.602987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943cbc3-334a-44c9-85f5-e23c7cff2560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:24.624195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0943cbc3-334a-44c9-85f5-e23c7cff2560 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:24.626205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0943cbc3-334a-44c9-85f5-e23c7cff2560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:24.699932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1281/2533] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:24.707611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d2bb3-9781-4774-abeb-492c6633cdb0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:24.707967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d2bb3-9781-4774-abeb-492c6633cdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:24.708312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d2bb3-9781-4774-abeb-492c6633cdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:24.727541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d43d2bb3-9781-4774-abeb-492c6633cdb0 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:24.728771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d43d2bb3-9781-4774-abeb-492c6633cdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:24.737684 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1253/2534] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:24.742344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943cbc3-334a-44c9-85f5-e23c7cff2560 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:24.752920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943cbc3-334a-44c9-85f5-e23c7cff2560 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:24.757782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:24.758468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:24.758749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:24.759025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:24.759797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:29:24.760105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:24.761736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:24.779388 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1282/2535] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:29:24.781018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d2bb3-9781-4774-abeb-492c6633cdb0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:24.794002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d2bb3-9781-4774-abeb-492c6633cdb0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:24.810549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9148e8-3ab9-4157-b11a-89fef6ffdc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:24.810549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9148e8-3ab9-4157-b11a-89fef6ffdc79 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:24.810549 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9148e8-3ab9-4157-b11a-89fef6ffdc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:24.811063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9148e8-3ab9-4157-b11a-89fef6ffdc79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:24.811997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9148e8-3ab9-4157-b11a-89fef6ffdc79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:24.814011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9148e8-3ab9-4157-b11a-89fef6ffdc79 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:24.851244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1254/2536] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:29:24 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:29:24.854640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:24.865313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770695a-4e02-4ab8-a9ac-03d01bdb83e8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:24.935369 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31fd69-92e1-426e-85df-3d64d9fea1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:24.936061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31fd69-92e1-426e-85df-3d64d9fea1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:24.936344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31fd69-92e1-426e-85df-3d64d9fea1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:24.936616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31fd69-92e1-426e-85df-3d64d9fea1f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:24.937371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31fd69-92e1-426e-85df-3d64d9fea1f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:24.939355 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31fd69-92e1-426e-85df-3d64d9fea1f1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:24.964123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1283/2537] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:29:24 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:29:24.965546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9148e8-3ab9-4157-b11a-89fef6ffdc79 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:24.976655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9148e8-3ab9-4157-b11a-89fef6ffdc79 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:24.981805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254daafc-4059-4d68-a826-6383c36a5770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:24.982513 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254daafc-4059-4d68-a826-6383c36a5770 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:24.982791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254daafc-4059-4d68-a826-6383c36a5770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:24.983063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254daafc-4059-4d68-a826-6383c36a5770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:24.983832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254daafc-4059-4d68-a826-6383c36a5770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:24.985483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254daafc-4059-4d68-a826-6383c36a5770 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:24.997644 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1255/2538] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:25.001565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31fd69-92e1-426e-85df-3d64d9fea1f1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:25.013247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31fd69-92e1-426e-85df-3d64d9fea1f1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:25.019611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de34e6-c9b7-4728-86af-8b2699fd4c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:25.020319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de34e6-c9b7-4728-86af-8b2699fd4c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:25.021034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de34e6-c9b7-4728-86af-8b2699fd4c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:25.021034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de34e6-c9b7-4728-86af-8b2699fd4c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:25.021573 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de34e6-c9b7-4728-86af-8b2699fd4c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:25.023921 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de34e6-c9b7-4728-86af-8b2699fd4c7f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:25.038546 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1284/2539] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:29:25.040053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254daafc-4059-4d68-a826-6383c36a5770 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:25.052518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254daafc-4059-4d68-a826-6383c36a5770 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:25.058632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:25.059665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:25.060033 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:25.060350 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] PATH_INFO: `/v3/projects/d3c238ac3f78461bb216cb9aa247b85e/users/bcb1d51d14274b3a919e1517c1eb0fa6/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:25.077877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c238ac3f78461bb216cb9aa247b85e, user_id=bcb1d51d14274b3a919e1517c1eb0fa6, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:25.080732 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:25.098587 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1256/2540] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:29:25 2022] PUT /identity/v3/projects/d3c238ac3f78461bb216cb9aa247b85e/users/bcb1d51d14274b3a919e1517c1eb0fa6/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:29:25.111670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3cc5-cf65-434e-a616-bab7aeee1178 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:25.111958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3cc5-cf65-434e-a616-bab7aeee1178 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:25.112248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3cc5-cf65-434e-a616-bab7aeee1178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:25.135695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:25.154415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7c0f3cc5-cf65-434e-a616-bab7aeee1178 None None] MFA Rules not processed for user `bcb1d51d14274b3a919e1517c1eb0fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:25.155689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7c0f3cc5-cf65-434e-a616-bab7aeee1178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:25.294983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1285/2541] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:25.308872 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:25.311284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:25.355672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:25.396852 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0584a66-3e40-4d76-937b-224f2fb5f73c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:25.400661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:25.401335 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:25.401613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:25.402042 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:25.411548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:25.412987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:25.429324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1257/2542] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:25.570712 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10697588-e094-4c27-9116-20febc745c52 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:25.571082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10697588-e094-4c27-9116-20febc745c52 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:25.571459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10697588-e094-4c27-9116-20febc745c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:25.596030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-10697588-e094-4c27-9116-20febc745c52 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:25.597757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10697588-e094-4c27-9116-20febc745c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:25.677833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1286/2543] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 08:29:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:29:25.755293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:25.757135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:25.872046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4b0ef-c022-4d4c-a70d-0efb5674b89f service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:25.888832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e1746-b7af-432d-a9f8-5089574adad1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:25.889815 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e1746-b7af-432d-a9f8-5089574adad1 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:25.890245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e1746-b7af-432d-a9f8-5089574adad1 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:25.891786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e1746-b7af-432d-a9f8-5089574adad1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:25.913621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e1746-b7af-432d-a9f8-5089574adad1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:25.915379 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e1746-b7af-432d-a9f8-5089574adad1 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:25.924760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1258/2544] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:29:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:29:28.669672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10697588-e094-4c27-9116-20febc745c52 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:28.736423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10697588-e094-4c27-9116-20febc745c52 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:28.742171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:28.743683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:28.743972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:28.744246 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] PATH_INFO: `/v3/users/bcb1d51d14274b3a919e1517c1eb0fa6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:28.745505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb1d51d14274b3a919e1517c1eb0fa6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:28.747482 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:28.800019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:28.801687 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb1d51d14274b3a919e1517c1eb0fa6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:28.802759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb1d51d14274b3a919e1517c1eb0fa6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:28.811140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb1d51d14274b3a919e1517c1eb0fa6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:28.823898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb1d51d14274b3a919e1517c1eb0fa6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:28.833681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb1d51d14274b3a919e1517c1eb0fa6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:28.840231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1287/2545] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:28 2022] DELETE /identity/v3/users/bcb1d51d14274b3a919e1517c1eb0fa6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:28.844949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58f083-7932-417d-b989-f092c63ccbda None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:28.845466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58f083-7932-417d-b989-f092c63ccbda None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:28.845466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58f083-7932-417d-b989-f092c63ccbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:28.863766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2d58f083-7932-417d-b989-f092c63ccbda None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:28.865126 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2d58f083-7932-417d-b989-f092c63ccbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:28.946745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1259/2546] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:28.959094 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:28.961098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:29.035505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d30f2-9887-46ec-82ac-9baf098490b2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:29.043440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:29.044543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:29.045067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:29.045427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:29.056962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:29.058559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:29.074188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1288/2547] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:29:29.643139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ebeac-2303-4f7b-a437-14d46f3f04cd None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:29.643711 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ebeac-2303-4f7b-a437-14d46f3f04cd None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:29.644058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ebeac-2303-4f7b-a437-14d46f3f04cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:29.661950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-817ebeac-2303-4f7b-a437-14d46f3f04cd None None] MFA Rules not processed for user `938efacddf794fd18cd323043906b6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:29.663610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-817ebeac-2303-4f7b-a437-14d46f3f04cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:29.747327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1260/2548] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:29.832392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:29.833805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:29.846329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035e09d-b12c-4260-8b6f-459752bf3ce0 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:29.850959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:29.851882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:29.852164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:29.852582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] PATH_INFO: `/v3/projects/d3c238ac3f78461bb216cb9aa247b85e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:29.853673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c238ac3f78461bb216cb9aa247b85e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:29.855700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:29.915477 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c238ac3f78461bb216cb9aa247b85e', 'request_id': 'req-10e95338-7dff-4b70-b33d-580693931768'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:29.920895 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c238ac3f78461bb216cb9aa247b85e', 'request_id': 'req-10e95338-7dff-4b70-b33d-580693931768'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:29.929396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c238ac3f78461bb216cb9aa247b85e', 'request_id': 'req-10e95338-7dff-4b70-b33d-580693931768'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:29.941272 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] The token cache is being invalidate because project d3c238ac3f78461bb216cb9aa247b85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:29.941557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:29.942223 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:29.943844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1289/2549] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:29 2022] DELETE /identity/v3/projects/d3c238ac3f78461bb216cb9aa247b85e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:34.031143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ebeac-2303-4f7b-a437-14d46f3f04cd None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:34.033145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-817ebeac-2303-4f7b-a437-14d46f3f04cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:34.048824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:34.066859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ebeac-2303-4f7b-a437-14d46f3f04cd None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:34.181856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:34.182972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:34.183521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:34.183860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] PATH_INFO: `/v3/users/c685a84382ec4f4fbd3be4378ba14ca9` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:34.185429 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c685a84382ec4f4fbd3be4378ba14ca9)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:34.188219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:34.267628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c685a84382ec4f4fbd3be4378ba14ca9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:34.278829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c685a84382ec4f4fbd3be4378ba14ca9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:34.297720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c685a84382ec4f4fbd3be4378ba14ca9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:34.298483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c685a84382ec4f4fbd3be4378ba14ca9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:34.312876 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c685a84382ec4f4fbd3be4378ba14ca9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:34.323259 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1261/2550] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:34 2022] DELETE /identity/v3/users/c685a84382ec4f4fbd3be4378ba14ca9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:34.932756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:34.934293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:34.947515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95338-7dff-4b70-b33d-580693931768 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:34.995756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:35.010449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:35.011490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:35.011772 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:35.012049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] PATH_INFO: `/v3/projects/796208f488e742d1beaef6d02f0232e6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:35.013267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=796208f488e742d1beaef6d02f0232e6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:35.015195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:35.062635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796208f488e742d1beaef6d02f0232e6', 'request_id': 'req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.068424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796208f488e742d1beaef6d02f0232e6', 'request_id': 'req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.077239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796208f488e742d1beaef6d02f0232e6', 'request_id': 'req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.088794 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] The token cache is being invalidate because project 796208f488e742d1beaef6d02f0232e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:35.089101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.089761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.091421 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1290/2551] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:34 2022] DELETE /identity/v3/projects/796208f488e742d1beaef6d02f0232e6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:35.095851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:35.097651 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:35.111441 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a8e03-4507-4a07-85a8-b676728e2c5d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:35.173330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:35.174223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:35.174499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:35.174767 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] PATH_INFO: `/v3/users/e7f719eee8e04bf9bfb4ba1d67a62c16` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:35.176171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f719eee8e04bf9bfb4ba1d67a62c16)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:35.178180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:35.209015 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:35.229787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f719eee8e04bf9bfb4ba1d67a62c16'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.238540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f719eee8e04bf9bfb4ba1d67a62c16'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.247589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f719eee8e04bf9bfb4ba1d67a62c16'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.248375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f719eee8e04bf9bfb4ba1d67a62c16'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.262652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5465147f-1002-4377-b0c3-6ea3b2e4874e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f719eee8e04bf9bfb4ba1d67a62c16'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.269026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1262/2552] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:29:35 2022] DELETE /identity/v3/users/e7f719eee8e04bf9bfb4ba1d67a62c16 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:35.495538 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:29:35.497165 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:35.516287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0503f0-94b5-4177-8505-8ab6e08c6e1b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:29:35.574162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:29:35.575073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:35.575392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:35.575660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] PATH_INFO: `/v3/projects/267d08d441a5422687e94066bb5c35f7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:35.577093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=267d08d441a5422687e94066bb5c35f7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:29:35.579048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:29:35.632911 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267d08d441a5422687e94066bb5c35f7', 'request_id': 'req-c2b71dd2-4462-456d-a033-d02411505768'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.638490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267d08d441a5422687e94066bb5c35f7', 'request_id': 'req-c2b71dd2-4462-456d-a033-d02411505768'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.647885 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267d08d441a5422687e94066bb5c35f7', 'request_id': 'req-c2b71dd2-4462-456d-a033-d02411505768'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.659525 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] The token cache is being invalidate because project 267d08d441a5422687e94066bb5c35f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:29:35.659792 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.660489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2b71dd2-4462-456d-a033-d02411505768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:29:35.662138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1291/2553] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:29:35 2022] DELETE /identity/v3/projects/267d08d441a5422687e94066bb5c35f7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:29:45.265334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=6,keystone:SELECT=19,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:45.656335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:29:47.907748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899483-ed61-4da1-a85f-e7eacd71e681 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:29:47.908290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899483-ed61-4da1-a85f-e7eacd71e681 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:29:47.908290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899483-ed61-4da1-a85f-e7eacd71e681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:29:47.927121 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ea899483-ed61-4da1-a85f-e7eacd71e681 None None] MFA Rules not processed for user `8d661ea344d74681bb7206f203033c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:29:47.928393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ea899483-ed61-4da1-a85f-e7eacd71e681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:29:47.986764 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1263/2554] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:29:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:29:57.979831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=25 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:30:00.481769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac26d2e-e178-4caf-8825-98c5a905698c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:00.482258 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac26d2e-e178-4caf-8825-98c5a905698c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:00.482353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac26d2e-e178-4caf-8825-98c5a905698c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:00.503727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7ac26d2e-e178-4caf-8825-98c5a905698c None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:30:00.505019 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ac26d2e-e178-4caf-8825-98c5a905698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:00.567561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1292/2555] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:30:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:30:00.579949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea899483-ed61-4da1-a85f-e7eacd71e681 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:30:00.581398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ea899483-ed61-4da1-a85f-e7eacd71e681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:00.659342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea899483-ed61-4da1-a85f-e7eacd71e681 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:30:00.664531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:30:00.665192 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:00.665475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:00.665751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:00.674017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:30:00.675661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:30:00.684078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1264/2556] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:30:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:30:10.560038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:30:10.681104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=32 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:30:11.104455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac26d2e-e178-4caf-8825-98c5a905698c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:30:11.106163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ac26d2e-e178-4caf-8825-98c5a905698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:11.180787 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac26d2e-e178-4caf-8825-98c5a905698c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:30:11.184467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:30:11.185126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:11.185415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:11.185694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:11.187768 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:11.278634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:30:11.280158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:30:11.294153 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1293/2557] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:30:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:30:14.860595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:30:14.862409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:14.875110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390266ca-031e-4953-8032-6be0d1a9d156 service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:30:14.879265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ef16a-f4a8-4476-989e-6d2a9c34a94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:30:14.879882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ef16a-f4a8-4476-989e-6d2a9c34a94c service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:14.880133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ef16a-f4a8-4476-989e-6d2a9c34a94c service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:14.880415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ef16a-f4a8-4476-989e-6d2a9c34a94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:14.882291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b57ef16a-f4a8-4476-989e-6d2a9c34a94c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:14.897305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ef16a-f4a8-4476-989e-6d2a9c34a94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:30:14.898743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ef16a-f4a8-4476-989e-6d2a9c34a94c service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:30:14.910322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1265/2558] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:30:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:30:18.374636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:30:18.376163 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:18.390543 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4970f2b-6e69-460e-81e9-194bbbc2010a service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:30:18.397555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:30:18.398122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:18.398405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:18.398909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:18.401049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:18.485801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:30:18.487654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:30:18.512251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1294/2559] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:30:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:30:19.123774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a73b6b-b143-479f-83c7-8aadbf46be2c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:19.124332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a73b6b-b143-479f-83c7-8aadbf46be2c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:19.124332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a73b6b-b143-479f-83c7-8aadbf46be2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:19.145877 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d4a73b6b-b143-479f-83c7-8aadbf46be2c None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:30:19.147021 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d4a73b6b-b143-479f-83c7-8aadbf46be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:19.155952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1266/2560] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 08:30:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:30:19.168622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:30:19.181630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45349f-e5de-4ecf-b150-5d99400cc11c service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:30:19.187403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7b553-e58f-4e15-8c10-a00a6c990225 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:30:19.188093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7b553-e58f-4e15-8c10-a00a6c990225 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:19.188376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7b553-e58f-4e15-8c10-a00a6c990225 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:19.188649 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7b553-e58f-4e15-8c10-a00a6c990225 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:19.197986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7b553-e58f-4e15-8c10-a00a6c990225 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:30:19.203232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7b553-e58f-4e15-8c10-a00a6c990225 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:30:19.203614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:30:19.211049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1295/2561] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:30:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:30:29.140360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=9 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:30:29.208439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:30:39.615616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a73b6b-b143-479f-83c7-8aadbf46be2c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:30:39.627073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a73b6b-b143-479f-83c7-8aadbf46be2c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:30:39.630751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:30:39.631613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:39.632080 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:39.632506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:39.634862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:39.647194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:30:39.648884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:30:39.657933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1267/2562] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:30:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:30:41.200068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661f27-3eab-48d6-8fc8-bb3f7c82b657 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:41.200651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661f27-3eab-48d6-8fc8-bb3f7c82b657 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:41.200651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661f27-3eab-48d6-8fc8-bb3f7c82b657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:41.222159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5f661f27-3eab-48d6-8fc8-bb3f7c82b657 None None] MFA Rules not processed for user `70d16599118b4968a18bf3f783a6687f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:30:41.223732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5f661f27-3eab-48d6-8fc8-bb3f7c82b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:30:41.231985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1296/2563] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 08:30:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:30:41.257350 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:30:41.273721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f72c3d-355c-4637-bbf6-adeb432ca124 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:30:41.280584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3481b5-7691-4cc1-8f25-60acc2df70ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7356a0f8d9f84e7cb785662db9afd658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:30:41.281316 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3481b5-7691-4cc1-8f25-60acc2df70ee service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:30:41.281603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3481b5-7691-4cc1-8f25-60acc2df70ee service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:30:41.281879 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3481b5-7691-4cc1-8f25-60acc2df70ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:30:41.293616 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3481b5-7691-4cc1-8f25-60acc2df70ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:30:41.295613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3481b5-7691-4cc1-8f25-60acc2df70ee service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:30:41.315408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1268/2564] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 08:30:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 08:30:51.219771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:30:51.304680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=12 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:28.186849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f661f27-3eab-48d6-8fc8-bb3f7c82b657 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:28.188574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5f661f27-3eab-48d6-8fc8-bb3f7c82b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:28.207274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f661f27-3eab-48d6-8fc8-bb3f7c82b657 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:28.216423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:28.217113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:28.217392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:28.217666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:28.220859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:28.238242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:28.241496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:28.259818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1297/2565] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:34.690401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3481b5-7691-4cc1-8f25-60acc2df70ee service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:34.700760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3481b5-7691-4cc1-8f25-60acc2df70ee service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:34.705540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:34.706218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:34.706509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:34.706785 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:34.709046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:34.777610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:34.778996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:34.789554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1269/2566] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:38.014714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:38.016865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:38.032801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12a9cd-1d60-42fe-9ff1-16503e26e7b2 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:38.037523 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:38.038431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.038708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.038974 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] PATH_INFO: `/v3/users/938efacddf794fd18cd323043906b6c1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.053761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] RBAC: Authorizing `identity:delete_user(user_id=938efacddf794fd18cd323043906b6c1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:38.056089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:38.104704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938efacddf794fd18cd323043906b6c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.105681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938efacddf794fd18cd323043906b6c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.113439 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938efacddf794fd18cd323043906b6c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.126500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938efacddf794fd18cd323043906b6c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.137267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6746ba94-6d1c-4eda-ab4f-f8d5d8e4aaba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938efacddf794fd18cd323043906b6c1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.145575 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1298/2567] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:31:38 2022] DELETE /identity/v3/users/938efacddf794fd18cd323043906b6c1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:38.328384 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:38.340290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc831e0-8769-471c-9e97-5a101b5a752f service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:38.404165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:38.405145 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.405435 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.405712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] PATH_INFO: `/v3/projects/4979d926d8894e3db358f16bd6bf7621` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.416561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4979d926d8894e3db358f16bd6bf7621)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:38.418508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:38.476214 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4979d926d8894e3db358f16bd6bf7621', 'request_id': 'req-558b2b8c-631d-4a8d-8c46-78b74365dbf9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.482461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4979d926d8894e3db358f16bd6bf7621', 'request_id': 'req-558b2b8c-631d-4a8d-8c46-78b74365dbf9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.492636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4979d926d8894e3db358f16bd6bf7621', 'request_id': 'req-558b2b8c-631d-4a8d-8c46-78b74365dbf9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.508361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] The token cache is being invalidate because project 4979d926d8894e3db358f16bd6bf7621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:31:38.508646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.509304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-558b2b8c-631d-4a8d-8c46-78b74365dbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:38.510887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1270/2568] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:31:38 2022] DELETE /identity/v3/projects/4979d926d8894e3db358f16bd6bf7621 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:38.516851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680eb158-895c-487e-9d16-f1becda6c5b0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.517344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680eb158-895c-487e-9d16-f1becda6c5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.517833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680eb158-895c-487e-9d16-f1becda6c5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.537027 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-680eb158-895c-487e-9d16-f1becda6c5b0 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:38.538152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-680eb158-895c-487e-9d16-f1becda6c5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:38.602048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1299/2569] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:38.610387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eff15-4883-4e95-be6b-3937d8d115d6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.610674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eff15-4883-4e95-be6b-3937d8d115d6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.610950 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eff15-4883-4e95-be6b-3937d8d115d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.630882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-182eff15-4883-4e95-be6b-3937d8d115d6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:38.632148 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-182eff15-4883-4e95-be6b-3937d8d115d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:38.641113 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1271/2570] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:38.645309 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680eb158-895c-487e-9d16-f1becda6c5b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:38.656070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680eb158-895c-487e-9d16-f1becda6c5b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:38.661239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:38.661900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.662179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.662446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.663174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:31:38.663511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:38.665133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:38.677176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1300/2571] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:31:38.681562 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eff15-4883-4e95-be6b-3937d8d115d6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:38.691512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eff15-4883-4e95-be6b-3937d8d115d6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:38.698009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda69d46-b486-4a9d-b25d-686bab9f6724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:38.698662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda69d46-b486-4a9d-b25d-686bab9f6724 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.698941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda69d46-b486-4a9d-b25d-686bab9f6724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.699229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda69d46-b486-4a9d-b25d-686bab9f6724 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.699928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda69d46-b486-4a9d-b25d-686bab9f6724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:38.701862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda69d46-b486-4a9d-b25d-686bab9f6724 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:38.706345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:38.731693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1272/2572] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:31:38 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:31:38.735920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:38.746611 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1f19a-106f-4605-b230-befb2758b9ee admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:38.787507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:38.816981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14b230-1c78-4546-8c9f-7669233c63ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:38.817651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b230-1c78-4546-8c9f-7669233c63ad admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.817927 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b230-1c78-4546-8c9f-7669233c63ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.818200 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b230-1c78-4546-8c9f-7669233c63ad admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.818883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14b230-1c78-4546-8c9f-7669233c63ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:38.820894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14b230-1c78-4546-8c9f-7669233c63ad admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:38.843295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1301/2573] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:31:38 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:38.847709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda69d46-b486-4a9d-b25d-686bab9f6724 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:38.858725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda69d46-b486-4a9d-b25d-686bab9f6724 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:38.864098 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8300f37-9666-42f3-8ff9-8d57571b7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:38.864788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300f37-9666-42f3-8ff9-8d57571b7e41 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.865386 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300f37-9666-42f3-8ff9-8d57571b7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.865661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300f37-9666-42f3-8ff9-8d57571b7e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.866396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8300f37-9666-42f3-8ff9-8d57571b7e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:38.868075 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8300f37-9666-42f3-8ff9-8d57571b7e41 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:38.879118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1273/2574] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:31:38.883435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14b230-1c78-4546-8c9f-7669233c63ad admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:38.894248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14b230-1c78-4546-8c9f-7669233c63ad admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:38.900415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971ec70-6f5e-4822-ad0c-0229e62b72e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:38.901109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971ec70-6f5e-4822-ad0c-0229e62b72e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.901394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971ec70-6f5e-4822-ad0c-0229e62b72e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.901670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971ec70-6f5e-4822-ad0c-0229e62b72e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.902434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971ec70-6f5e-4822-ad0c-0229e62b72e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:38.904118 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971ec70-6f5e-4822-ad0c-0229e62b72e5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:38.914761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1302/2575] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:38.919189 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8300f37-9666-42f3-8ff9-8d57571b7e41 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:38.929991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8300f37-9666-42f3-8ff9-8d57571b7e41 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:38.935979 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:38.936881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.937170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.937437 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] PATH_INFO: `/v3/projects/fb09d404c2c547bdba1beeacd86e69e2/users/259b1784639a471bbbcf4cd06fca1cde/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:38.953478 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb09d404c2c547bdba1beeacd86e69e2, user_id=259b1784639a471bbbcf4cd06fca1cde, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:38.956168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:38.973131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1274/2576] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:38 2022] PUT /identity/v3/projects/fb09d404c2c547bdba1beeacd86e69e2/users/259b1784639a471bbbcf4cd06fca1cde/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:38.983836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c7d79-d1c1-4199-b20c-967def963366 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:38.984121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c7d79-d1c1-4199-b20c-967def963366 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:38.984399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c7d79-d1c1-4199-b20c-967def963366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:39.012845 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4d0c7d79-d1c1-4199-b20c-967def963366 None None] MFA Rules not processed for user `259b1784639a471bbbcf4cd06fca1cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:39.014065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4d0c7d79-d1c1-4199-b20c-967def963366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:39.082825 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1303/2577] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:39.095299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:39.097974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:39.183681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9c90b-a6a6-4616-b8b8-e59a9cf421e7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:39.190410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc55ad-7108-4cee-9941-d4e50e668e3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:39.191070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc55ad-7108-4cee-9941-d4e50e668e3b service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:39.191404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc55ad-7108-4cee-9941-d4e50e668e3b service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:39.191677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc55ad-7108-4cee-9941-d4e50e668e3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:39.199784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc55ad-7108-4cee-9941-d4e50e668e3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:39.201388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc55ad-7108-4cee-9941-d4e50e668e3b service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:39.210788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1275/2578] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:39.435595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c7d79-d1c1-4199-b20c-967def963366 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:39.515059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c7d79-d1c1-4199-b20c-967def963366 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:39.520320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:39.521212 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:39.521492 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:39.521764 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] PATH_INFO: `/v3/users/259b1784639a471bbbcf4cd06fca1cde` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:39.523013 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=259b1784639a471bbbcf4cd06fca1cde)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:39.524989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:39.575784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:39.577444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '259b1784639a471bbbcf4cd06fca1cde'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:39.578277 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '259b1784639a471bbbcf4cd06fca1cde'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:39.586526 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '259b1784639a471bbbcf4cd06fca1cde'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:39.599241 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '259b1784639a471bbbcf4cd06fca1cde'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:39.610031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '259b1784639a471bbbcf4cd06fca1cde'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:39.617767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1304/2579] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:31:39 2022] DELETE /identity/v3/users/259b1784639a471bbbcf4cd06fca1cde => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:39.624060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa613141-adfa-46b3-8de7-47a944893980 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:39.624561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa613141-adfa-46b3-8de7-47a944893980 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:39.624670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa613141-adfa-46b3-8de7-47a944893980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:39.644955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fa613141-adfa-46b3-8de7-47a944893980 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:39.646172 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fa613141-adfa-46b3-8de7-47a944893980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:39.708114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1276/2580] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:31:39.718029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:39.719910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:39.785437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f17a3-5d94-4f45-a3e9-b6127255a720 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:39.790389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae61ae6-8468-4906-833d-e7977c9c66b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:39.791068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61ae6-8468-4906-833d-e7977c9c66b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:39.791522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61ae6-8468-4906-833d-e7977c9c66b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:39.791791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61ae6-8468-4906-833d-e7977c9c66b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:39.802423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae61ae6-8468-4906-833d-e7977c9c66b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:39.803919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae61ae6-8468-4906-833d-e7977c9c66b7 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:39.813871 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1305/2581] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:40.118939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa613141-adfa-46b3-8de7-47a944893980 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.120552 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fa613141-adfa-46b3-8de7-47a944893980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:40.138674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa613141-adfa-46b3-8de7-47a944893980 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:40.144638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:40.145471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.145759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.146036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] PATH_INFO: `/v3/projects/fb09d404c2c547bdba1beeacd86e69e2` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.147257 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb09d404c2c547bdba1beeacd86e69e2)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:40.149197 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:40.187593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:40.201667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb09d404c2c547bdba1beeacd86e69e2', 'request_id': 'req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:40.207690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb09d404c2c547bdba1beeacd86e69e2', 'request_id': 'req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:40.216610 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb09d404c2c547bdba1beeacd86e69e2', 'request_id': 'req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:40.228060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] The token cache is being invalidate because project fb09d404c2c547bdba1beeacd86e69e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:31:40.228345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:40.229093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f32fa147-2fd3-4ee0-9ce4-a3d9335ecbcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:40.230751 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1277/2582] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:31:40 2022] DELETE /identity/v3/projects/fb09d404c2c547bdba1beeacd86e69e2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:40.236021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbaabf-3c10-4101-9ea4-c13de9ee04aa None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.236496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbaabf-3c10-4101-9ea4-c13de9ee04aa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.236589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbaabf-3c10-4101-9ea4-c13de9ee04aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.254727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8acbaabf-3c10-4101-9ea4-c13de9ee04aa None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:40.255996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8acbaabf-3c10-4101-9ea4-c13de9ee04aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:40.318639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1306/2583] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:40.327947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b4b0f-943d-4ded-a3a7-5cb8f437fa1d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.328242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b4b0f-943d-4ded-a3a7-5cb8f437fa1d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.328638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b4b0f-943d-4ded-a3a7-5cb8f437fa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.350255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ff5b4b0f-943d-4ded-a3a7-5cb8f437fa1d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:40.350992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ff5b4b0f-943d-4ded-a3a7-5cb8f437fa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:40.361124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1278/2584] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:40.366006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbaabf-3c10-4101-9ea4-c13de9ee04aa None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.376779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbaabf-3c10-4101-9ea4-c13de9ee04aa None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:40.382023 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:40.382689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.382970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.383259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.384008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:31:40.384318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:40.385972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:40.398476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1307/2585] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:31:40.402925 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b4b0f-943d-4ded-a3a7-5cb8f437fa1d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.413377 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b4b0f-943d-4ded-a3a7-5cb8f437fa1d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:40.418940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356624b-8547-42d3-acad-e2d306683255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:40.419652 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356624b-8547-42d3-acad-e2d306683255 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.419933 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356624b-8547-42d3-acad-e2d306683255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.420206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356624b-8547-42d3-acad-e2d306683255 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.420923 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356624b-8547-42d3-acad-e2d306683255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:40.423526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356624b-8547-42d3-acad-e2d306683255 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:40.452721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1279/2586] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:31:40 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:40.457252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.468112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db674e-6083-4109-bd39-d362277df53a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:40.520066 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:40.529985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f26263-74a0-417a-ae64-1079de1236f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:40.530657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26263-74a0-417a-ae64-1079de1236f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.530941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26263-74a0-417a-ae64-1079de1236f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.531240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26263-74a0-417a-ae64-1079de1236f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.532365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f26263-74a0-417a-ae64-1079de1236f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:40.534814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f26263-74a0-417a-ae64-1079de1236f5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:40.556795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1308/2587] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:31:40 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:40.560987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356624b-8547-42d3-acad-e2d306683255 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.572973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356624b-8547-42d3-acad-e2d306683255 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:40.578612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0d7ec-bf31-4aed-a0a6-463b55873da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:40.579720 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0d7ec-bf31-4aed-a0a6-463b55873da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.580008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0d7ec-bf31-4aed-a0a6-463b55873da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.580284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0d7ec-bf31-4aed-a0a6-463b55873da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.581025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0d7ec-bf31-4aed-a0a6-463b55873da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:40.582667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0d7ec-bf31-4aed-a0a6-463b55873da3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:40.593828 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1280/2588] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:40.598463 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f26263-74a0-417a-ae64-1079de1236f5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.609253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f26263-74a0-417a-ae64-1079de1236f5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:40.615236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a80a9-04ab-46f4-b83f-12664f0f5484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:40.615896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a80a9-04ab-46f4-b83f-12664f0f5484 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.616195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a80a9-04ab-46f4-b83f-12664f0f5484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.616487 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a80a9-04ab-46f4-b83f-12664f0f5484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.617216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a80a9-04ab-46f4-b83f-12664f0f5484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:40.618836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a80a9-04ab-46f4-b83f-12664f0f5484 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:40.629747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1309/2589] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:40.635343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0d7ec-bf31-4aed-a0a6-463b55873da3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.647012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0d7ec-bf31-4aed-a0a6-463b55873da3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:40.652682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652b2bd-d3f2-4297-b0f5-f68ec7e18485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:40.653574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b2bd-d3f2-4297-b0f5-f68ec7e18485 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.653867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b2bd-d3f2-4297-b0f5-f68ec7e18485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.654144 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b2bd-d3f2-4297-b0f5-f68ec7e18485 admin admin] PATH_INFO: `/v3/projects/93155e59b3f34c4b801ac6c87c56fbfc/users/f617315600c949e6b6cb9ad006454984/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.670731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b2bd-d3f2-4297-b0f5-f68ec7e18485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93155e59b3f34c4b801ac6c87c56fbfc, user_id=f617315600c949e6b6cb9ad006454984, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:40.673671 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b2bd-d3f2-4297-b0f5-f68ec7e18485 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:40.690039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1281/2590] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:40 2022] PUT /identity/v3/projects/93155e59b3f34c4b801ac6c87c56fbfc/users/f617315600c949e6b6cb9ad006454984/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:40.700583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d9f9a-d2c9-4278-81e1-f1c842cdae20 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.700910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d9f9a-d2c9-4278-81e1-f1c842cdae20 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.701187 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d9f9a-d2c9-4278-81e1-f1c842cdae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.728920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b72d9f9a-d2c9-4278-81e1-f1c842cdae20 None None] MFA Rules not processed for user `f617315600c949e6b6cb9ad006454984`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:40.730022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b72d9f9a-d2c9-4278-81e1-f1c842cdae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:40.802166 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1310/2591] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:40 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:40.806869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652b2bd-d3f2-4297-b0f5-f68ec7e18485 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.875492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652b2bd-d3f2-4297-b0f5-f68ec7e18485 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:40.881150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8672fee-b953-4f22-a685-d5a197d0e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:40.881819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8672fee-b953-4f22-a685-d5a197d0e0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:40.882093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8672fee-b953-4f22-a685-d5a197d0e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:40.882365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8672fee-b953-4f22-a685-d5a197d0e0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:40.883065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8672fee-b953-4f22-a685-d5a197d0e0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:40.885022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8672fee-b953-4f22-a685-d5a197d0e0aa admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:40.918967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1282/2592] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:31:40 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:40.924121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d9f9a-d2c9-4278-81e1-f1c842cdae20 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:40.941521 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d9f9a-d2c9-4278-81e1-f1c842cdae20 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:41.020416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7ba13-61f4-40d9-bdae-7d02e90d3d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:41.021007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ba13-61f4-40d9-bdae-7d02e90d3d7c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.021289 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ba13-61f4-40d9-bdae-7d02e90d3d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.021566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ba13-61f4-40d9-bdae-7d02e90d3d7c admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.022248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7ba13-61f4-40d9-bdae-7d02e90d3d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:41.024222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7ba13-61f4-40d9-bdae-7d02e90d3d7c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:41.048831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1311/2593] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:31:40 2022] POST /identity/v3/users => generated 308 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:41.053211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8672fee-b953-4f22-a685-d5a197d0e0aa admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:41.064786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8672fee-b953-4f22-a685-d5a197d0e0aa admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:41.070248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977bbe7-146b-4c6f-95ae-9908776bbb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:41.070917 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977bbe7-146b-4c6f-95ae-9908776bbb64 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.071228 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977bbe7-146b-4c6f-95ae-9908776bbb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.071514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977bbe7-146b-4c6f-95ae-9908776bbb64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.072252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977bbe7-146b-4c6f-95ae-9908776bbb64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:41.073905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977bbe7-146b-4c6f-95ae-9908776bbb64 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:41.086059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1283/2594] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:41.090180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7ba13-61f4-40d9-bdae-7d02e90d3d7c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:41.100702 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7ba13-61f4-40d9-bdae-7d02e90d3d7c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:41.106587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb5fe7-e3a2-4eda-92a9-4aff0bb0462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:41.107527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5fe7-e3a2-4eda-92a9-4aff0bb0462b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.107814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5fe7-e3a2-4eda-92a9-4aff0bb0462b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.108089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5fe7-e3a2-4eda-92a9-4aff0bb0462b admin admin] PATH_INFO: `/v3/projects/2ccfb5ae75a2478488f23e5b6647988b/users/2c7d942e99c54ec89dfbbd630a574156/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.132558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb5fe7-e3a2-4eda-92a9-4aff0bb0462b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccfb5ae75a2478488f23e5b6647988b, user_id=2c7d942e99c54ec89dfbbd630a574156, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:41.136859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb5fe7-e3a2-4eda-92a9-4aff0bb0462b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:41.157566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1312/2595] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:41 2022] PUT /identity/v3/projects/2ccfb5ae75a2478488f23e5b6647988b/users/2c7d942e99c54ec89dfbbd630a574156/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:41.162956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977bbe7-146b-4c6f-95ae-9908776bbb64 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:41.176222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977bbe7-146b-4c6f-95ae-9908776bbb64 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:41.210678 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:41.242740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b559-c6c6-4939-b757-59b90db02ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:41.243440 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b559-c6c6-4939-b757-59b90db02ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.243722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b559-c6c6-4939-b757-59b90db02ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.243994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b559-c6c6-4939-b757-59b90db02ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.244718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867b559-c6c6-4939-b757-59b90db02ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:41.246315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867b559-c6c6-4939-b757-59b90db02ab9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:41.256243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1284/2596] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:41.260469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb5fe7-e3a2-4eda-92a9-4aff0bb0462b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:41.272158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb5fe7-e3a2-4eda-92a9-4aff0bb0462b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:41.277207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:41.278086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.278368 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.278638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] PATH_INFO: `/v3/projects/2ccfb5ae75a2478488f23e5b6647988b/users/2c7d942e99c54ec89dfbbd630a574156/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.280708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccfb5ae75a2478488f23e5b6647988b, user_id=2c7d942e99c54ec89dfbbd630a574156, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:41.283581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:41.294555 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1313/2597] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:41 2022] PUT /identity/v3/projects/2ccfb5ae75a2478488f23e5b6647988b/users/2c7d942e99c54ec89dfbbd630a574156/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:31:41.304666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfb915-e2f9-435c-8142-5ba5292ca1f9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.304952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfb915-e2f9-435c-8142-5ba5292ca1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.305224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfb915-e2f9-435c-8142-5ba5292ca1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.338231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f4dfb915-e2f9-435c-8142-5ba5292ca1f9 None None] MFA Rules not processed for user `2c7d942e99c54ec89dfbbd630a574156`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:41.339518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f4dfb915-e2f9-435c-8142-5ba5292ca1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:41.414465 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1285/2598] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:41.425966 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:41.428327 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:41.445304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:41.496264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462b68b-4ccd-4703-96a0-da2d80981df5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:41.500111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade0f7d-a6a5-4a21-b080-a57f00707422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:41.501067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0f7d-a6a5-4a21-b080-a57f00707422 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.501349 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0f7d-a6a5-4a21-b080-a57f00707422 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.501609 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0f7d-a6a5-4a21-b080-a57f00707422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.511941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade0f7d-a6a5-4a21-b080-a57f00707422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:41.513500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade0f7d-a6a5-4a21-b080-a57f00707422 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:41.523903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1314/2599] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:41 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:41.676009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfb915-e2f9-435c-8142-5ba5292ca1f9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:41.741458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfb915-e2f9-435c-8142-5ba5292ca1f9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:41.746505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:41.747475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.747755 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.748027 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] PATH_INFO: `/v3/users/f617315600c949e6b6cb9ad006454984` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.749295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f617315600c949e6b6cb9ad006454984)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:41.751641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:41.802566 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f617315600c949e6b6cb9ad006454984'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:41.810746 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f617315600c949e6b6cb9ad006454984'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:41.812177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:41.820224 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f617315600c949e6b6cb9ad006454984'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:41.821115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f617315600c949e6b6cb9ad006454984'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:41.833076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f617315600c949e6b6cb9ad006454984'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:41.839310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1286/2600] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:31:41 2022] DELETE /identity/v3/users/f617315600c949e6b6cb9ad006454984 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:41.843958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23d82e-707c-4bdb-8839-d5071b03797c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:41.844418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23d82e-707c-4bdb-8839-d5071b03797c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:41.844698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23d82e-707c-4bdb-8839-d5071b03797c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:41.864252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7d23d82e-707c-4bdb-8839-d5071b03797c None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:41.865761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d23d82e-707c-4bdb-8839-d5071b03797c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:41.928987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1315/2601] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:41.939074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:41.940787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:42.010434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f82aa3-666a-435e-a612-4ae66899ea70 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:42.014819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99559c-368d-4ec4-a292-f28eec046952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:42.015500 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99559c-368d-4ec4-a292-f28eec046952 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:42.015777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99559c-368d-4ec4-a292-f28eec046952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:42.016043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99559c-368d-4ec4-a292-f28eec046952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:42.025890 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99559c-368d-4ec4-a292-f28eec046952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:42.027356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99559c-368d-4ec4-a292-f28eec046952 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:42.037861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1287/2602] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:42.325247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23d82e-707c-4bdb-8839-d5071b03797c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:42.326706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d23d82e-707c-4bdb-8839-d5071b03797c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:42.340961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23d82e-707c-4bdb-8839-d5071b03797c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:42.347043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:42.348302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:42.348586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:42.348886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] PATH_INFO: `/v3/projects/93155e59b3f34c4b801ac6c87c56fbfc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:42.350186 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=93155e59b3f34c4b801ac6c87c56fbfc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:42.352249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:42.405285 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93155e59b3f34c4b801ac6c87c56fbfc', 'request_id': 'req-e4abf952-6da0-4b91-8656-1e98969d28fd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.411638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93155e59b3f34c4b801ac6c87c56fbfc', 'request_id': 'req-e4abf952-6da0-4b91-8656-1e98969d28fd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.420929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93155e59b3f34c4b801ac6c87c56fbfc', 'request_id': 'req-e4abf952-6da0-4b91-8656-1e98969d28fd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.432679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] The token cache is being invalidate because project 93155e59b3f34c4b801ac6c87c56fbfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:31:42.432970 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.433606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.435182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1316/2603] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:31:42 2022] DELETE /identity/v3/projects/93155e59b3f34c4b801ac6c87c56fbfc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:42.439771 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99559c-368d-4ec4-a292-f28eec046952 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:42.449487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99559c-368d-4ec4-a292-f28eec046952 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:42.508983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:42.510003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:42.510154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:42.510417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] PATH_INFO: `/v3/users/2c7d942e99c54ec89dfbbd630a574156` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:42.511689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c7d942e99c54ec89dfbbd630a574156)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:42.513656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:42.564296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c7d942e99c54ec89dfbbd630a574156'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.573783 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c7d942e99c54ec89dfbbd630a574156'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.581360 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c7d942e99c54ec89dfbbd630a574156'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.582065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c7d942e99c54ec89dfbbd630a574156'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.593063 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:42.595985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f3bd8efc-b1b9-482f-8e96-d14581cd18e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c7d942e99c54ec89dfbbd630a574156'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:42.604206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1288/2604] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:31:42 2022] DELETE /identity/v3/users/2c7d942e99c54ec89dfbbd630a574156 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:42.888686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:42.890189 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:42.903741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf952-6da0-4b91-8656-1e98969d28fd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:42.942683 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:42.963370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:42.964430 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:42.964712 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:42.964983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] PATH_INFO: `/v3/projects/2ccfb5ae75a2478488f23e5b6647988b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:42.966174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccfb5ae75a2478488f23e5b6647988b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:42.968135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.017539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccfb5ae75a2478488f23e5b6647988b', 'request_id': 'req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:43.029615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccfb5ae75a2478488f23e5b6647988b', 'request_id': 'req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:43.041801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccfb5ae75a2478488f23e5b6647988b', 'request_id': 'req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:43.053470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] The token cache is being invalidate because project 2ccfb5ae75a2478488f23e5b6647988b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:31:43.053752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:43.054410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f2bba2eb-7ab3-45c1-b83f-d68cb6e2e8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:43.056055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1317/2605] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:31:42 2022] DELETE /identity/v3/projects/2ccfb5ae75a2478488f23e5b6647988b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:43.061356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fc827-eb7c-4c4b-bd9e-8103b2f89ca5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.061897 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fc827-eb7c-4c4b-bd9e-8103b2f89ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.062016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fc827-eb7c-4c4b-bd9e-8103b2f89ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.081729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8d5fc827-eb7c-4c4b-bd9e-8103b2f89ca5 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:43.082947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8d5fc827-eb7c-4c4b-bd9e-8103b2f89ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:43.150218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1289/2606] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:31:43.158320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102ad5-11bf-4ccf-a593-1638e2ca412e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.158612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102ad5-11bf-4ccf-a593-1638e2ca412e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.158888 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102ad5-11bf-4ccf-a593-1638e2ca412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.177874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-89102ad5-11bf-4ccf-a593-1638e2ca412e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:43.179075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-89102ad5-11bf-4ccf-a593-1638e2ca412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:43.187565 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1318/2607] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:43.191943 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fc827-eb7c-4c4b-bd9e-8103b2f89ca5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.202373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fc827-eb7c-4c4b-bd9e-8103b2f89ca5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.208538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.209201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.209479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.209747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.210483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:31:43.210788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.212431 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.224713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1290/2608] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:31:43.229011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89102ad5-11bf-4ccf-a593-1638e2ca412e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.238485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89102ad5-11bf-4ccf-a593-1638e2ca412e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.244312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb1c27-677c-4a42-b1ac-a2b59da569a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.244987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb1c27-677c-4a42-b1ac-a2b59da569a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.245240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb1c27-677c-4a42-b1ac-a2b59da569a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.245508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb1c27-677c-4a42-b1ac-a2b59da569a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.246208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb1c27-677c-4a42-b1ac-a2b59da569a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.248245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb1c27-677c-4a42-b1ac-a2b59da569a3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.281423 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1319/2609] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:31:43 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:31:43.286322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.300535 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee88e7e-c5fe-4308-b0af-be2f68ad96e5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.362893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037bbc7-6f2f-4a7b-841f-4f1a1fb38a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.363752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037bbc7-6f2f-4a7b-841f-4f1a1fb38a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.364037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037bbc7-6f2f-4a7b-841f-4f1a1fb38a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.364312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037bbc7-6f2f-4a7b-841f-4f1a1fb38a7c admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.365344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037bbc7-6f2f-4a7b-841f-4f1a1fb38a7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.367272 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037bbc7-6f2f-4a7b-841f-4f1a1fb38a7c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.390788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1291/2610] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:31:43 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:43.395483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb1c27-677c-4a42-b1ac-a2b59da569a3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.406286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb1c27-677c-4a42-b1ac-a2b59da569a3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.412038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe7a4f-bea4-4a5e-a1c8-d8f501010c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.412706 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe7a4f-bea4-4a5e-a1c8-d8f501010c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.412980 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe7a4f-bea4-4a5e-a1c8-d8f501010c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.413253 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe7a4f-bea4-4a5e-a1c8-d8f501010c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.413973 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe7a4f-bea4-4a5e-a1c8-d8f501010c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.415668 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe7a4f-bea4-4a5e-a1c8-d8f501010c97 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.427470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1320/2611] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:43.431568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037bbc7-6f2f-4a7b-841f-4f1a1fb38a7c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.442504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037bbc7-6f2f-4a7b-841f-4f1a1fb38a7c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.448786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307642c2-5f76-45c6-98ab-0cdb8dacd44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.449452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307642c2-5f76-45c6-98ab-0cdb8dacd44b admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.449731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307642c2-5f76-45c6-98ab-0cdb8dacd44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.450000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307642c2-5f76-45c6-98ab-0cdb8dacd44b admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.450731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307642c2-5f76-45c6-98ab-0cdb8dacd44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.452362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307642c2-5f76-45c6-98ab-0cdb8dacd44b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.463189 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1292/2612] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:43.467418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe7a4f-bea4-4a5e-a1c8-d8f501010c97 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.477677 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe7a4f-bea4-4a5e-a1c8-d8f501010c97 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.482907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ed2f1-a72c-418d-ba27-f6d4e3c9adea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.483825 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ed2f1-a72c-418d-ba27-f6d4e3c9adea admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.484104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ed2f1-a72c-418d-ba27-f6d4e3c9adea admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.484380 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ed2f1-a72c-418d-ba27-f6d4e3c9adea admin admin] PATH_INFO: `/v3/projects/a29dad66063444d8844a766ae097a940/users/34d3517a7db4495aaa2664d695a1b493/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.500204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ed2f1-a72c-418d-ba27-f6d4e3c9adea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29dad66063444d8844a766ae097a940, user_id=34d3517a7db4495aaa2664d695a1b493, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.503011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ed2f1-a72c-418d-ba27-f6d4e3c9adea admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.517435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1321/2613] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:43 2022] PUT /identity/v3/projects/a29dad66063444d8844a766ae097a940/users/34d3517a7db4495aaa2664d695a1b493/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:43.528000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75f5ff-e370-4e35-ba1e-af5592966fea None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.528289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75f5ff-e370-4e35-ba1e-af5592966fea None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.528591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75f5ff-e370-4e35-ba1e-af5592966fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.557081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fd75f5ff-e370-4e35-ba1e-af5592966fea None None] MFA Rules not processed for user `34d3517a7db4495aaa2664d695a1b493`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:43.558539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fd75f5ff-e370-4e35-ba1e-af5592966fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:43.605010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:43.626092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1293/2614] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:31:43.630601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ed2f1-a72c-418d-ba27-f6d4e3c9adea admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.693344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ed2f1-a72c-418d-ba27-f6d4e3c9adea admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.698207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1b2d-09b3-49ee-9d1c-3a930d3848cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.698860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1b2d-09b3-49ee-9d1c-3a930d3848cd admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.699142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1b2d-09b3-49ee-9d1c-3a930d3848cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.699442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1b2d-09b3-49ee-9d1c-3a930d3848cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.700134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d1b2d-09b3-49ee-9d1c-3a930d3848cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.702047 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d1b2d-09b3-49ee-9d1c-3a930d3848cd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.717160 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:43.740313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1322/2615] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:31:43 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:43.744489 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75f5ff-e370-4e35-ba1e-af5592966fea None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.755930 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75f5ff-e370-4e35-ba1e-af5592966fea None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.819466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2abf-9a37-4d76-966a-6bdc5643d0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.820155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2abf-9a37-4d76-966a-6bdc5643d0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.820530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2abf-9a37-4d76-966a-6bdc5643d0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.820814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2abf-9a37-4d76-966a-6bdc5643d0a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.821532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c2abf-9a37-4d76-966a-6bdc5643d0a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.823495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c2abf-9a37-4d76-966a-6bdc5643d0a0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.848611 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1294/2616] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:31:43 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:43.852762 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1b2d-09b3-49ee-9d1c-3a930d3848cd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.864029 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1b2d-09b3-49ee-9d1c-3a930d3848cd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.869761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80827a-005b-4952-99ae-7a2470dd7eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.870424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80827a-005b-4952-99ae-7a2470dd7eda admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.870701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80827a-005b-4952-99ae-7a2470dd7eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.870968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80827a-005b-4952-99ae-7a2470dd7eda admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.871769 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80827a-005b-4952-99ae-7a2470dd7eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.873410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80827a-005b-4952-99ae-7a2470dd7eda admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.884688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1323/2617] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:43.889020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2abf-9a37-4d76-966a-6bdc5643d0a0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.899090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2abf-9a37-4d76-966a-6bdc5643d0a0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:43.903742 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a942e2e-3a23-4e3f-b24c-08a53a7c71e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:43.904651 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a942e2e-3a23-4e3f-b24c-08a53a7c71e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:43.904932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a942e2e-3a23-4e3f-b24c-08a53a7c71e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:43.905201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a942e2e-3a23-4e3f-b24c-08a53a7c71e4 admin admin] PATH_INFO: `/v3/projects/5bed59fae951465cacbff8658b7f0ef5/users/b511fd74d8334f0fbfc5bb82ae21f149/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:43.921329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a942e2e-3a23-4e3f-b24c-08a53a7c71e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bed59fae951465cacbff8658b7f0ef5, user_id=b511fd74d8334f0fbfc5bb82ae21f149, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:43.924258 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a942e2e-3a23-4e3f-b24c-08a53a7c71e4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:43.941143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1295/2618] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:43 2022] PUT /identity/v3/projects/5bed59fae951465cacbff8658b7f0ef5/users/b511fd74d8334f0fbfc5bb82ae21f149/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:43.945248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80827a-005b-4952-99ae-7a2470dd7eda admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:43.955550 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80827a-005b-4952-99ae-7a2470dd7eda admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:44.016057 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ed73-e733-47fe-9769-6edb0c5d8990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:44.016932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ed73-e733-47fe-9769-6edb0c5d8990 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:44.017216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ed73-e733-47fe-9769-6edb0c5d8990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:44.017453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ed73-e733-47fe-9769-6edb0c5d8990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:44.018291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ed73-e733-47fe-9769-6edb0c5d8990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:44.019997 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ed73-e733-47fe-9769-6edb0c5d8990 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:44.030083 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1324/2619] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:44.035191 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a942e2e-3a23-4e3f-b24c-08a53a7c71e4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:44.046653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a942e2e-3a23-4e3f-b24c-08a53a7c71e4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:44.052456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:44.053396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:44.053670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:44.053922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] PATH_INFO: `/v3/projects/5bed59fae951465cacbff8658b7f0ef5/users/b511fd74d8334f0fbfc5bb82ae21f149/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:44.055980 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bed59fae951465cacbff8658b7f0ef5, user_id=b511fd74d8334f0fbfc5bb82ae21f149, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:44.058620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:44.069759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1296/2620] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:44 2022] PUT /identity/v3/projects/5bed59fae951465cacbff8658b7f0ef5/users/b511fd74d8334f0fbfc5bb82ae21f149/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:44.082514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea112b-7b84-4391-b25f-d2cda2b067a9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:44.082863 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea112b-7b84-4391-b25f-d2cda2b067a9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:44.083267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea112b-7b84-4391-b25f-d2cda2b067a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:44.115912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32ea112b-7b84-4391-b25f-d2cda2b067a9 None None] MFA Rules not processed for user `b511fd74d8334f0fbfc5bb82ae21f149`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:44.117142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32ea112b-7b84-4391-b25f-d2cda2b067a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:44.199416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1325/2621] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:44.213814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:44.216116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:44.278663 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39c6eb-a742-457a-98ce-d16265ea165c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:44.282687 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04ec89-6b59-4bb2-8427-7da949245e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:44.283373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ec89-6b59-4bb2-8427-7da949245e72 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:44.283709 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ec89-6b59-4bb2-8427-7da949245e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:44.283988 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ec89-6b59-4bb2-8427-7da949245e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:44.293716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04ec89-6b59-4bb2-8427-7da949245e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:44.295189 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04ec89-6b59-4bb2-8427-7da949245e72 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:44.304844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1297/2622] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:44.390728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea112b-7b84-4391-b25f-d2cda2b067a9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:44.401182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea112b-7b84-4391-b25f-d2cda2b067a9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:44.405073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4f7e0-b33e-45ae-9cd6-dc847f6359de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:44.405733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4f7e0-b33e-45ae-9cd6-dc847f6359de service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:44.406009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4f7e0-b33e-45ae-9cd6-dc847f6359de service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:44.406282 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4f7e0-b33e-45ae-9cd6-dc847f6359de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:44.467642 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4f7e0-b33e-45ae-9cd6-dc847f6359de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:44.469018 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4f7e0-b33e-45ae-9cd6-dc847f6359de service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:44.473767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:44.478581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1326/2623] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:47.217536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04ec89-6b59-4bb2-8427-7da949245e72 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:47.265001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:47.283067 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04ec89-6b59-4bb2-8427-7da949245e72 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:47.287622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:47.288507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:47.288789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:47.289065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] PATH_INFO: `/v3/users/34d3517a7db4495aaa2664d695a1b493` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:47.290245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d3517a7db4495aaa2664d695a1b493)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:47.292225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:47.341341 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d3517a7db4495aaa2664d695a1b493'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.349715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d3517a7db4495aaa2664d695a1b493'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.357060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d3517a7db4495aaa2664d695a1b493'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.357784 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d3517a7db4495aaa2664d695a1b493'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.371130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d3517a7db4495aaa2664d695a1b493'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.377134 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1298/2624] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:31:47 2022] DELETE /identity/v3/users/34d3517a7db4495aaa2664d695a1b493 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:47.381818 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576c702-6284-4b49-8ab6-b9f8511cd728 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:47.382326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576c702-6284-4b49-8ab6-b9f8511cd728 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:47.382422 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576c702-6284-4b49-8ab6-b9f8511cd728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:47.400079 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0576c702-6284-4b49-8ab6-b9f8511cd728 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:47.401286 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0576c702-6284-4b49-8ab6-b9f8511cd728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:47.474963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1327/2625] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:47.487995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:47.489707 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:47.551048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb637e-0ca0-4133-b071-c339dd533780 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:47.556300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628315b-a0d7-453e-ba68-a955bb448855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:47.556951 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628315b-a0d7-453e-ba68-a955bb448855 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:47.557229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628315b-a0d7-453e-ba68-a955bb448855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:47.557502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628315b-a0d7-453e-ba68-a955bb448855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:47.566926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628315b-a0d7-453e-ba68-a955bb448855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:47.568399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628315b-a0d7-453e-ba68-a955bb448855 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:47.577827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1299/2626] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:47.869954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576c702-6284-4b49-8ab6-b9f8511cd728 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:47.871673 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0576c702-6284-4b49-8ab6-b9f8511cd728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:47.885635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576c702-6284-4b49-8ab6-b9f8511cd728 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:47.890996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:47.891924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:47.892208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:47.892483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] PATH_INFO: `/v3/projects/a29dad66063444d8844a766ae097a940` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:47.894058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29dad66063444d8844a766ae097a940)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:47.896132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:47.944367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29dad66063444d8844a766ae097a940', 'request_id': 'req-462aa031-52a5-4190-99bd-089e7bb29769'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.950856 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29dad66063444d8844a766ae097a940', 'request_id': 'req-462aa031-52a5-4190-99bd-089e7bb29769'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.959537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29dad66063444d8844a766ae097a940', 'request_id': 'req-462aa031-52a5-4190-99bd-089e7bb29769'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.971387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] The token cache is being invalidate because project a29dad66063444d8844a766ae097a940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:31:47.971692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.972424 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:47.974080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1328/2627] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:31:47 2022] DELETE /identity/v3/projects/a29dad66063444d8844a766ae097a940 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:47.978137 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628315b-a0d7-453e-ba68-a955bb448855 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:47.988279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628315b-a0d7-453e-ba68-a955bb448855 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:48.047419 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:48.048339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.048788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.049077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] PATH_INFO: `/v3/users/b511fd74d8334f0fbfc5bb82ae21f149` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.050413 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b511fd74d8334f0fbfc5bb82ae21f149)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:48.052551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:48.058543 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:48.112499 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b511fd74d8334f0fbfc5bb82ae21f149'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.122572 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b511fd74d8334f0fbfc5bb82ae21f149'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.129630 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b511fd74d8334f0fbfc5bb82ae21f149'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.130269 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b511fd74d8334f0fbfc5bb82ae21f149'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.142091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9e39a5ba-7428-49eb-9dc2-bccba702ee2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b511fd74d8334f0fbfc5bb82ae21f149'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.148831 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1300/2628] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:31:47 2022] DELETE /identity/v3/users/b511fd74d8334f0fbfc5bb82ae21f149 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:48.436455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:48.438060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:48.451815 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa031-52a5-4190-99bd-089e7bb29769 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:48.511335 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:48.512219 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.512500 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.512775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] PATH_INFO: `/v3/projects/5bed59fae951465cacbff8658b7f0ef5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.513976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bed59fae951465cacbff8658b7f0ef5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:48.516017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:48.549091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:48.569458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bed59fae951465cacbff8658b7f0ef5', 'request_id': 'req-2b430607-6e35-4f43-b3dd-4306237c8c9a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.574812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bed59fae951465cacbff8658b7f0ef5', 'request_id': 'req-2b430607-6e35-4f43-b3dd-4306237c8c9a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.583632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bed59fae951465cacbff8658b7f0ef5', 'request_id': 'req-2b430607-6e35-4f43-b3dd-4306237c8c9a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.594533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] The token cache is being invalidate because project 5bed59fae951465cacbff8658b7f0ef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:31:48.594819 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.595534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b430607-6e35-4f43-b3dd-4306237c8c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:31:48.597131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1329/2629] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:31:48 2022] DELETE /identity/v3/projects/5bed59fae951465cacbff8658b7f0ef5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:48.602970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6c544-8391-4fa2-921f-c7503fa70fa4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.603541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6c544-8391-4fa2-921f-c7503fa70fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.603648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6c544-8391-4fa2-921f-c7503fa70fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.630410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-15a6c544-8391-4fa2-921f-c7503fa70fa4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:48.631615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-15a6c544-8391-4fa2-921f-c7503fa70fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:48.696845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1301/2630] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:48.704986 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8d43c-cc67-460e-a152-f675bfca667e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.705345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8d43c-cc67-460e-a152-f675bfca667e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.705570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8d43c-cc67-460e-a152-f675bfca667e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.725306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6cf8d43c-cc67-460e-a152-f675bfca667e None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:48.726515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6cf8d43c-cc67-460e-a152-f675bfca667e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:48.734759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1330/2631] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:48.739178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6c544-8391-4fa2-921f-c7503fa70fa4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:48.749599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6c544-8391-4fa2-921f-c7503fa70fa4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:48.754699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:48.755375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.755690 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.755931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.756745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:31:48.757058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:48.758665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:48.770544 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1302/2632] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:31:48.775263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8d43c-cc67-460e-a152-f675bfca667e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:48.788944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8d43c-cc67-460e-a152-f675bfca667e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:48.796044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6da15-f5e6-4754-84c2-94502893f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:48.796704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6da15-f5e6-4754-84c2-94502893f2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.796992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6da15-f5e6-4754-84c2-94502893f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.797265 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6da15-f5e6-4754-84c2-94502893f2c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.797977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6da15-f5e6-4754-84c2-94502893f2c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:48.800415 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6da15-f5e6-4754-84c2-94502893f2c6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:48.829312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1331/2633] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:31:48 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:48.833765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:48.843972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cc37e-1a3b-4613-8c5c-daa1bf77be89 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:48.906068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249e7d5-590a-410e-b9e8-a8e9ca9196c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:48.906656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e7d5-590a-410e-b9e8-a8e9ca9196c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.906936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e7d5-590a-410e-b9e8-a8e9ca9196c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.907235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e7d5-590a-410e-b9e8-a8e9ca9196c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.908013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249e7d5-590a-410e-b9e8-a8e9ca9196c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:48.910095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249e7d5-590a-410e-b9e8-a8e9ca9196c0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:48.932696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1303/2634] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:31:48 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:48.938254 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6da15-f5e6-4754-84c2-94502893f2c6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:48.949420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6da15-f5e6-4754-84c2-94502893f2c6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:48.955657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb175c-81ca-4bff-b9d3-40ccab0b7a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:48.956343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb175c-81ca-4bff-b9d3-40ccab0b7a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.956621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb175c-81ca-4bff-b9d3-40ccab0b7a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.956890 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb175c-81ca-4bff-b9d3-40ccab0b7a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.957679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb175c-81ca-4bff-b9d3-40ccab0b7a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:48.959338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb175c-81ca-4bff-b9d3-40ccab0b7a61 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:48.970585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1332/2635] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:48.974628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249e7d5-590a-410e-b9e8-a8e9ca9196c0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:48.980172 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:48.987053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249e7d5-590a-410e-b9e8-a8e9ca9196c0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:48.994032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045adf6-b114-4905-8d27-1ddacfbc0c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:48.994716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045adf6-b114-4905-8d27-1ddacfbc0c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:48.994995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045adf6-b114-4905-8d27-1ddacfbc0c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:48.995296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045adf6-b114-4905-8d27-1ddacfbc0c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:48.996029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045adf6-b114-4905-8d27-1ddacfbc0c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:48.997665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045adf6-b114-4905-8d27-1ddacfbc0c27 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.008344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1304/2636] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:49.012638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb175c-81ca-4bff-b9d3-40ccab0b7a61 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:49.022961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb175c-81ca-4bff-b9d3-40ccab0b7a61 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:49.028457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6158fc8-90d8-46be-8f40-d8940068194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:49.029376 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6158fc8-90d8-46be-8f40-d8940068194a admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.029653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6158fc8-90d8-46be-8f40-d8940068194a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.029921 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6158fc8-90d8-46be-8f40-d8940068194a admin admin] PATH_INFO: `/v3/projects/23aea789193c449390fca808e5475b0f/users/e9d3298571cd4a318b6e25d49f30569d/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.050503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6158fc8-90d8-46be-8f40-d8940068194a admin admin] RBAC: Authorizing `identity:create_grant(project_id=23aea789193c449390fca808e5475b0f, user_id=e9d3298571cd4a318b6e25d49f30569d, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:49.053225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6158fc8-90d8-46be-8f40-d8940068194a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.069334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1333/2637] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:49 2022] PUT /identity/v3/projects/23aea789193c449390fca808e5475b0f/users/e9d3298571cd4a318b6e25d49f30569d/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:31:49.082295 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41589dee-bdf0-41f9-9044-26c9d5501f7c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.082589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41589dee-bdf0-41f9-9044-26c9d5501f7c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.082871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41589dee-bdf0-41f9-9044-26c9d5501f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.110900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-41589dee-bdf0-41f9-9044-26c9d5501f7c None None] MFA Rules not processed for user `e9d3298571cd4a318b6e25d49f30569d`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:49.112202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-41589dee-bdf0-41f9-9044-26c9d5501f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:49.179800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1305/2638] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:49.184736 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6158fc8-90d8-46be-8f40-d8940068194a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:49.252669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6158fc8-90d8-46be-8f40-d8940068194a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:49.257840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b00e0b-2f66-4dd2-bb6c-e3f96e043281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:49.258544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00e0b-2f66-4dd2-bb6c-e3f96e043281 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.258826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00e0b-2f66-4dd2-bb6c-e3f96e043281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.259111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00e0b-2f66-4dd2-bb6c-e3f96e043281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.259831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b00e0b-2f66-4dd2-bb6c-e3f96e043281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:49.261770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b00e0b-2f66-4dd2-bb6c-e3f96e043281 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.293595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1334/2639] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:31:49 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:49.297912 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41589dee-bdf0-41f9-9044-26c9d5501f7c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:49.309039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41589dee-bdf0-41f9-9044-26c9d5501f7c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:49.367334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343561f-47e0-44d2-8d70-ef6b6eb19454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:49.368022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343561f-47e0-44d2-8d70-ef6b6eb19454 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.368306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343561f-47e0-44d2-8d70-ef6b6eb19454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.368579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343561f-47e0-44d2-8d70-ef6b6eb19454 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.369307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343561f-47e0-44d2-8d70-ef6b6eb19454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:49.371264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343561f-47e0-44d2-8d70-ef6b6eb19454 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.393865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1306/2640] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:31:49 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:31:49.397866 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b00e0b-2f66-4dd2-bb6c-e3f96e043281 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:49.408656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b00e0b-2f66-4dd2-bb6c-e3f96e043281 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:49.413767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a38642-ef50-4659-b46a-14391e8cd12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:49.414436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38642-ef50-4659-b46a-14391e8cd12a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.414718 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38642-ef50-4659-b46a-14391e8cd12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.414994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38642-ef50-4659-b46a-14391e8cd12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.415767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a38642-ef50-4659-b46a-14391e8cd12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:49.417382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a38642-ef50-4659-b46a-14391e8cd12a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.428304 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1335/2641] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:49.432704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343561f-47e0-44d2-8d70-ef6b6eb19454 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:49.448073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343561f-47e0-44d2-8d70-ef6b6eb19454 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:49.454011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d09c7-5545-456f-9ef3-173bbaf806be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:49.455068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d09c7-5545-456f-9ef3-173bbaf806be admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.455390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d09c7-5545-456f-9ef3-173bbaf806be admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.455666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d09c7-5545-456f-9ef3-173bbaf806be admin admin] PATH_INFO: `/v3/projects/e7cd251cc5ae4a2aa5168ea6acad4174/users/c82997fb0f9c408992f484cfd19904ba/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.471481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d09c7-5545-456f-9ef3-173bbaf806be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7cd251cc5ae4a2aa5168ea6acad4174, user_id=c82997fb0f9c408992f484cfd19904ba, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:49.474400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d09c7-5545-456f-9ef3-173bbaf806be admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.499654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1307/2642] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:49 2022] PUT /identity/v3/projects/e7cd251cc5ae4a2aa5168ea6acad4174/users/c82997fb0f9c408992f484cfd19904ba/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:49.503512 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a38642-ef50-4659-b46a-14391e8cd12a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:49.515587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a38642-ef50-4659-b46a-14391e8cd12a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:49.536121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:49.578050 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fbe6a-a008-4a78-8579-bf1e542a1197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:49.578732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fbe6a-a008-4a78-8579-bf1e542a1197 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.579017 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fbe6a-a008-4a78-8579-bf1e542a1197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.579356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fbe6a-a008-4a78-8579-bf1e542a1197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.580205 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fbe6a-a008-4a78-8579-bf1e542a1197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:49.581902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fbe6a-a008-4a78-8579-bf1e542a1197 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.593106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1336/2643] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:31:49.597397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d09c7-5545-456f-9ef3-173bbaf806be admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:49.612083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d09c7-5545-456f-9ef3-173bbaf806be admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:49.618059 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:49.619034 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.619430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.619768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] PATH_INFO: `/v3/projects/e7cd251cc5ae4a2aa5168ea6acad4174/users/c82997fb0f9c408992f484cfd19904ba/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.621806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7cd251cc5ae4a2aa5168ea6acad4174, user_id=c82997fb0f9c408992f484cfd19904ba, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:49.624836 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.636293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1308/2644] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:31:49 2022] PUT /identity/v3/projects/e7cd251cc5ae4a2aa5168ea6acad4174/users/c82997fb0f9c408992f484cfd19904ba/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:31:49.648081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce96eb1-6cf4-4f14-8fee-71afbe0aaf4f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.648441 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce96eb1-6cf4-4f14-8fee-71afbe0aaf4f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.648725 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce96eb1-6cf4-4f14-8fee-71afbe0aaf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.677294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7ce96eb1-6cf4-4f14-8fee-71afbe0aaf4f None None] MFA Rules not processed for user `c82997fb0f9c408992f484cfd19904ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:31:49.678533 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ce96eb1-6cf4-4f14-8fee-71afbe0aaf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:49.752775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1337/2645] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:31:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:31:49.765506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:49.767449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:31:49.804253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:31:49.829203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9252f-bf5b-4e72-9cfb-2950d4892939 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:49.833153 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:49.833814 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:49.834099 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:49.834374 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:49.894579 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:49.899693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:49.905201 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1309/2646] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:56.602446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce96eb1-6cf4-4f14-8fee-71afbe0aaf4f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:31:56.613383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce96eb1-6cf4-4f14-8fee-71afbe0aaf4f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:31:56.618178 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:31:56.618910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:31:56.619195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:31:56.619498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:31:56.629891 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:31:56.631406 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:31:56.642073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1338/2647] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:31:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:31:59.901453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=33 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:03.899576 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:03.901271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:03.966753 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef2c49-5613-4655-ad5f-19fac17a7ff2 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:03.972020 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:03.972795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:03.973076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:03.973348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:03.976088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:04.048600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:04.049985 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:04.060084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1310/2648] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:06.637042 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=58 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:07.432065 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:07.433704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:07.451552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7e0e6-5f2c-4048-bc21-74c0eb8d7eb9 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:07.456164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:07.457061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:07.457345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:07.457765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] PATH_INFO: `/v3/users/9b178568bc5e4876bdab388ac3302f0a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:07.458971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b178568bc5e4876bdab388ac3302f0a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:07.461684 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:07.513135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b178568bc5e4876bdab388ac3302f0a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.513822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b178568bc5e4876bdab388ac3302f0a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.520932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b178568bc5e4876bdab388ac3302f0a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.534006 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b178568bc5e4876bdab388ac3302f0a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.543784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3805826b-ef1f-432b-826d-d250fc91955b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b178568bc5e4876bdab388ac3302f0a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.550109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1339/2649] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:32:07 2022] DELETE /identity/v3/users/9b178568bc5e4876bdab388ac3302f0a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:07.744615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:07.755140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce743777-f608-4bd2-b68e-c9a70f2ec946 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:07.819453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:07.820393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:07.820677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:07.820954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] PATH_INFO: `/v3/projects/89774ed6e4964866bf6fd73e32dee267` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:07.822235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=89774ed6e4964866bf6fd73e32dee267)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:07.824315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:07.878795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89774ed6e4964866bf6fd73e32dee267', 'request_id': 'req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.881024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:07.885900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89774ed6e4964866bf6fd73e32dee267', 'request_id': 'req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.897084 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89774ed6e4964866bf6fd73e32dee267', 'request_id': 'req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.909220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] The token cache is being invalidate because project 89774ed6e4964866bf6fd73e32dee267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:32:07.909443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.910057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8be2ffc6-b6fc-4b16-acce-00cf2c961c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:07.911668 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1311/2650] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:32:07 2022] DELETE /identity/v3/projects/89774ed6e4964866bf6fd73e32dee267 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:32:07.917261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b12f9c-864b-4810-af64-0ebb273c99b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:07.917820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b12f9c-864b-4810-af64-0ebb273c99b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:07.918025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b12f9c-864b-4810-af64-0ebb273c99b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:07.936967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a3b12f9c-864b-4810-af64-0ebb273c99b1 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:07.938185 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a3b12f9c-864b-4810-af64-0ebb273c99b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:07.998131 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1340/2651] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:08.006650 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa65e1c-c44f-4a0c-988c-506c3285c48b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.007008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa65e1c-c44f-4a0c-988c-506c3285c48b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.007504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa65e1c-c44f-4a0c-988c-506c3285c48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.026452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-baa65e1c-c44f-4a0c-988c-506c3285c48b None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:08.027694 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-baa65e1c-c44f-4a0c-988c-506c3285c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:08.035632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1312/2652] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:08.040115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b12f9c-864b-4810-af64-0ebb273c99b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:08.050312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b12f9c-864b-4810-af64-0ebb273c99b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:08.056012 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:08.056714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.056994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.057266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.058030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:32:08.058334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:08.060001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:08.072553 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1341/2653] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:32:08.077216 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa65e1c-c44f-4a0c-988c-506c3285c48b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:08.087757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa65e1c-c44f-4a0c-988c-506c3285c48b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:08.093427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fa564-6350-435d-83ca-1f11d3912040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:08.094077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fa564-6350-435d-83ca-1f11d3912040 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.094353 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fa564-6350-435d-83ca-1f11d3912040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.094715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fa564-6350-435d-83ca-1f11d3912040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.095506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fa564-6350-435d-83ca-1f11d3912040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:08.097548 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fa564-6350-435d-83ca-1f11d3912040 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:08.127501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1313/2654] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:32:08 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:32:08.131841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:08.142788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444ee9b-54c4-410e-99b8-3c7dac8aa856 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:08.209208 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b881fc-56cc-4a61-abdd-196bebe7cdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:08.209959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b881fc-56cc-4a61-abdd-196bebe7cdb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.210318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b881fc-56cc-4a61-abdd-196bebe7cdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.210589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b881fc-56cc-4a61-abdd-196bebe7cdb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.211446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b881fc-56cc-4a61-abdd-196bebe7cdb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:08.213396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b881fc-56cc-4a61-abdd-196bebe7cdb8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:08.220408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:08.236801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1342/2655] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:32:08 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:32:08.241064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fa564-6350-435d-83ca-1f11d3912040 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:08.251937 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fa564-6350-435d-83ca-1f11d3912040 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:08.256882 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838c9f-ded8-437b-96d1-3fd929052f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:08.257737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838c9f-ded8-437b-96d1-3fd929052f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.258019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838c9f-ded8-437b-96d1-3fd929052f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.258294 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838c9f-ded8-437b-96d1-3fd929052f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.259050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3838c9f-ded8-437b-96d1-3fd929052f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:08.260689 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3838c9f-ded8-437b-96d1-3fd929052f43 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:08.271845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1314/2656] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:08.275960 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b881fc-56cc-4a61-abdd-196bebe7cdb8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:08.286084 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b881fc-56cc-4a61-abdd-196bebe7cdb8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:08.291104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be06da-34e2-4598-ac19-a52d153ab31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:08.291802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be06da-34e2-4598-ac19-a52d153ab31e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.292082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be06da-34e2-4598-ac19-a52d153ab31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.292354 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be06da-34e2-4598-ac19-a52d153ab31e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.293081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be06da-34e2-4598-ac19-a52d153ab31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:08.294695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be06da-34e2-4598-ac19-a52d153ab31e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:08.304730 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1343/2657] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:08.308924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838c9f-ded8-437b-96d1-3fd929052f43 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:08.318927 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838c9f-ded8-437b-96d1-3fd929052f43 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:08.323778 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:08.324665 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.324946 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.325217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] PATH_INFO: `/v3/projects/e27862b1b0f2434d86d6af9998716432/users/4ab1ccac328d47c99119b8205cabfedd/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.339722 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27862b1b0f2434d86d6af9998716432, user_id=4ab1ccac328d47c99119b8205cabfedd, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:08.342347 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:08.361654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1315/2658] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:32:08 2022] PUT /identity/v3/projects/e27862b1b0f2434d86d6af9998716432/users/4ab1ccac328d47c99119b8205cabfedd/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:08.369669 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ef75-fcf2-412d-bc10-85a4af9713f4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.370082 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ef75-fcf2-412d-bc10-85a4af9713f4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.370433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ef75-fcf2-412d-bc10-85a4af9713f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.398126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7430ef75-fcf2-412d-bc10-85a4af9713f4 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:08.399322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7430ef75-fcf2-412d-bc10-85a4af9713f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:08.461908 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1344/2659] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:08.471474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:08.473301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:08.535473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5cedf-f4bb-4965-8349-9de27ad5e855 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:08.540114 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e60f3-9be7-4752-bb39-6e71ccdc1794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:08.540735 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e60f3-9be7-4752-bb39-6e71ccdc1794 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.541012 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e60f3-9be7-4752-bb39-6e71ccdc1794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.541281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e60f3-9be7-4752-bb39-6e71ccdc1794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.550563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e60f3-9be7-4752-bb39-6e71ccdc1794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:08.552206 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e60f3-9be7-4752-bb39-6e71ccdc1794 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:08.563916 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1316/2660] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:08.792046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430ef75-fcf2-412d-bc10-85a4af9713f4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:08.793638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7430ef75-fcf2-412d-bc10-85a4af9713f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:08.865161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430ef75-fcf2-412d-bc10-85a4af9713f4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:08.870410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:08.871224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:08.871574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:08.871909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:08.879883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:08.954425 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:08.956837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:08.967754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1345/2661] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:12.651285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9cfe-209f-4bd5-94f9-061f41c04f7a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:12.651803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9cfe-209f-4bd5-94f9-061f41c04f7a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:12.651900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9cfe-209f-4bd5-94f9-061f41c04f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:12.686283 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b11a9cfe-209f-4bd5-94f9-061f41c04f7a None None] MFA Rules not processed for user `4ab1ccac328d47c99119b8205cabfedd`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:12.687536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b11a9cfe-209f-4bd5-94f9-061f41c04f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:12.759479 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1317/2662] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:12.775437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:12.777157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:12.814987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:12.856608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b8ab0-c3b6-475c-922b-603e64cb5214 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:12.862504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:12.863176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:12.863470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:12.863744 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:12.873465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:12.875097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:12.885054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1346/2663] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:14.179715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f465099-a3d9-4e3e-a5c5-1e0ede5de654 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:14.181362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f465099-a3d9-4e3e-a5c5-1e0ede5de654 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:14.181642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f465099-a3d9-4e3e-a5c5-1e0ede5de654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:14.201178 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7f465099-a3d9-4e3e-a5c5-1e0ede5de654 None None] MFA Rules not processed for user `4ab1ccac328d47c99119b8205cabfedd`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:14.202396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7f465099-a3d9-4e3e-a5c5-1e0ede5de654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:14.214550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1318/2664] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:14.577161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:14.579793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:14.595297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc5eab-918e-41a5-bb59-84b83042102b service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:14.599468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0f15d-c10d-4f46-9e15-ef63bec030a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:14.600311 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0f15d-c10d-4f46-9e15-ef63bec030a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:14.600949 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0f15d-c10d-4f46-9e15-ef63bec030a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:14.601235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0f15d-c10d-4f46-9e15-ef63bec030a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:14.604694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48f0f15d-c10d-4f46-9e15-ef63bec030a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:14.615487 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0f15d-c10d-4f46-9e15-ef63bec030a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:14.617587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0f15d-c10d-4f46-9e15-ef63bec030a2 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:14.626558 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1347/2665] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:15.063152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f465099-a3d9-4e3e-a5c5-1e0ede5de654 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:15.065261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7f465099-a3d9-4e3e-a5c5-1e0ede5de654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:15.071287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:15.193758 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f465099-a3d9-4e3e-a5c5-1e0ede5de654 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:15.200221 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:15.201387 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:15.201837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:15.202289 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:15.204680 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:15.220370 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:15.222273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:15.239074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1319/2666] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:24.360161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf3c5-f954-4067-82f7-fd15a3ebdc20 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:24.360737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf3c5-f954-4067-82f7-fd15a3ebdc20 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:24.360737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf3c5-f954-4067-82f7-fd15a3ebdc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:24.378968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0d4cf3c5-f954-4067-82f7-fd15a3ebdc20 None None] MFA Rules not processed for user `8d661ea344d74681bb7206f203033c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:24.380092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0d4cf3c5-f954-4067-82f7-fd15a3ebdc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:24.387496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1348/2667] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:24.707858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:24.718666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fe836-c5bb-418e-95c5-88ce7fa377ce service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:24.723251 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:24.723906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:24.724183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:24.724454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:24.726554 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:24.736655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:24.738118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:24.746628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1320/2668] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:30.491459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf3c5-f954-4067-82f7-fd15a3ebdc20 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:30.492950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0d4cf3c5-f954-4067-82f7-fd15a3ebdc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:30.515454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf3c5-f954-4067-82f7-fd15a3ebdc20 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:30.521916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9faa5f-1c4d-4b42-9e24-5c9e10b0d049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:30.522716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9faa5f-1c4d-4b42-9e24-5c9e10b0d049 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:30.523251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9faa5f-1c4d-4b42-9e24-5c9e10b0d049 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:30.523527 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9faa5f-1c4d-4b42-9e24-5c9e10b0d049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:30.525943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bc9faa5f-1c4d-4b42-9e24-5c9e10b0d049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:30.540471 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9faa5f-1c4d-4b42-9e24-5c9e10b0d049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:30.542704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9faa5f-1c4d-4b42-9e24-5c9e10b0d049 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:30.555117 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1349/2669] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:32.185866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:32.187455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:32.202822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7737-f9cb-4998-9d0c-a3699917821c service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:32.209001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:32.209891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:32.210169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:32.210439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] PATH_INFO: `/v3/users/e9d3298571cd4a318b6e25d49f30569d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:32.211899 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9d3298571cd4a318b6e25d49f30569d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:32.213832 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:32.266531 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9d3298571cd4a318b6e25d49f30569d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:32.276140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9d3298571cd4a318b6e25d49f30569d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:32.283344 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9d3298571cd4a318b6e25d49f30569d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:32.284010 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9d3298571cd4a318b6e25d49f30569d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:32.295973 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9d3298571cd4a318b6e25d49f30569d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:32.301865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1321/2670] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:32:32 2022] DELETE /identity/v3/users/e9d3298571cd4a318b6e25d49f30569d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:32.308711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024305d-6e1c-4475-8418-213c698c2546 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:32.309235 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024305d-6e1c-4475-8418-213c698c2546 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:32.309345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024305d-6e1c-4475-8418-213c698c2546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:32.333034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3024305d-6e1c-4475-8418-213c698c2546 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:32.334231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3024305d-6e1c-4475-8418-213c698c2546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:32.396656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1350/2671] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:32.406455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:32.408081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:32.466538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef1d51-96da-4289-968d-af70e9dcfd1c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:32.470512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92315209-f585-4310-b38a-367d7a5d5068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:32.471158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315209-f585-4310-b38a-367d7a5d5068 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:32.471474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315209-f585-4310-b38a-367d7a5d5068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:32.471745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315209-f585-4310-b38a-367d7a5d5068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:32.477847 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:32.482474 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92315209-f585-4310-b38a-367d7a5d5068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:32.484455 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92315209-f585-4310-b38a-367d7a5d5068 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:32.496532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1322/2672] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:32.896254 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024305d-6e1c-4475-8418-213c698c2546 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:32.897783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3024305d-6e1c-4475-8418-213c698c2546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:32.910457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024305d-6e1c-4475-8418-213c698c2546 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:32.916216 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:32.917106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:32.917387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:32.917665 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] PATH_INFO: `/v3/projects/23aea789193c449390fca808e5475b0f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:32.918951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23aea789193c449390fca808e5475b0f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:32.920941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:32.977679 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23aea789193c449390fca808e5475b0f', 'request_id': 'req-a40e15a2-094d-4f57-bbd4-00bbd7642e55'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:32.984249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23aea789193c449390fca808e5475b0f', 'request_id': 'req-a40e15a2-094d-4f57-bbd4-00bbd7642e55'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:32.993640 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23aea789193c449390fca808e5475b0f', 'request_id': 'req-a40e15a2-094d-4f57-bbd4-00bbd7642e55'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.004941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] The token cache is being invalidate because project 23aea789193c449390fca808e5475b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:32:33.005224 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.005860 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.007499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1351/2673] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:32:32 2022] DELETE /identity/v3/projects/23aea789193c449390fca808e5475b0f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:33.011612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92315209-f585-4310-b38a-367d7a5d5068 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:33.021627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92315209-f585-4310-b38a-367d7a5d5068 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:33.092170 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:33.092974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:33.093253 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:33.093522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] PATH_INFO: `/v3/users/c82997fb0f9c408992f484cfd19904ba` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:33.094754 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82997fb0f9c408992f484cfd19904ba)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:33.096749 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:33.149921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82997fb0f9c408992f484cfd19904ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.159893 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82997fb0f9c408992f484cfd19904ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.166835 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82997fb0f9c408992f484cfd19904ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.167509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82997fb0f9c408992f484cfd19904ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.180975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-770d3dad-4ee3-4c52-a86e-cb949513fd16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82997fb0f9c408992f484cfd19904ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.187358 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1323/2674] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:32:33 2022] DELETE /identity/v3/users/c82997fb0f9c408992f484cfd19904ba => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:33.557561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:33.559038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:33.573680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e15a2-094d-4f57-bbd4-00bbd7642e55 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:33.579083 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:33.635616 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:33.636486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:33.636756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:33.637030 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] PATH_INFO: `/v3/projects/e7cd251cc5ae4a2aa5168ea6acad4174` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:33.638210 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7cd251cc5ae4a2aa5168ea6acad4174)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:33.640230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:33.697515 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7cd251cc5ae4a2aa5168ea6acad4174', 'request_id': 'req-c238f0d7-7e2b-483b-81ac-f609b5404a3d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.704428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7cd251cc5ae4a2aa5168ea6acad4174', 'request_id': 'req-c238f0d7-7e2b-483b-81ac-f609b5404a3d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.715621 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7cd251cc5ae4a2aa5168ea6acad4174', 'request_id': 'req-c238f0d7-7e2b-483b-81ac-f609b5404a3d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.727662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] The token cache is being invalidate because project e7cd251cc5ae4a2aa5168ea6acad4174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:32:33.728037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.728692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c238f0d7-7e2b-483b-81ac-f609b5404a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:33.730303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1352/2675] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:32:33 2022] DELETE /identity/v3/projects/e7cd251cc5ae4a2aa5168ea6acad4174 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:33.736071 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91d7aa-9d97-4226-a55f-07dea54ae9b9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:33.736366 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91d7aa-9d97-4226-a55f-07dea54ae9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:33.736649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91d7aa-9d97-4226-a55f-07dea54ae9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:33.756325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9b91d7aa-9d97-4226-a55f-07dea54ae9b9 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:33.757573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9b91d7aa-9d97-4226-a55f-07dea54ae9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:33.821016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1324/2676] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:33.829220 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902269eb-e9fa-4252-89b5-b0f94fc406fb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:33.829504 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902269eb-e9fa-4252-89b5-b0f94fc406fb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:33.829775 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902269eb-e9fa-4252-89b5-b0f94fc406fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:33.848957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-902269eb-e9fa-4252-89b5-b0f94fc406fb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:33.850162 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-902269eb-e9fa-4252-89b5-b0f94fc406fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:33.858823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1353/2677] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:33.863504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91d7aa-9d97-4226-a55f-07dea54ae9b9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:33.873529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91d7aa-9d97-4226-a55f-07dea54ae9b9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:33.878239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:33.878911 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:33.879194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:33.879502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:33.880271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:32:33.880575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:33.882183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:33.891538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:33.895708 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1325/2678] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:32:33.899485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902269eb-e9fa-4252-89b5-b0f94fc406fb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:33.909108 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902269eb-e9fa-4252-89b5-b0f94fc406fb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:33.915851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1da17-8867-4c11-ae02-0c06fcf37940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:33.916582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1da17-8867-4c11-ae02-0c06fcf37940 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:33.916847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1da17-8867-4c11-ae02-0c06fcf37940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:33.917139 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1da17-8867-4c11-ae02-0c06fcf37940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:33.917805 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1da17-8867-4c11-ae02-0c06fcf37940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:33.919820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1da17-8867-4c11-ae02-0c06fcf37940 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:33.949601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1354/2679] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:32:33 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:32:33.954122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:33.964115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bb4d7-4517-4fd4-a9cb-d4e23e3dc555 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.022960 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c462c0-8638-4e18-9436-74e328ec714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.023681 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c462c0-8638-4e18-9436-74e328ec714c admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.023954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c462c0-8638-4e18-9436-74e328ec714c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.024217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c462c0-8638-4e18-9436-74e328ec714c admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.024931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c462c0-8638-4e18-9436-74e328ec714c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.026892 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c462c0-8638-4e18-9436-74e328ec714c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.050850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1326/2680] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:32:33 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:32:34.055140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1da17-8867-4c11-ae02-0c06fcf37940 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.066745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1da17-8867-4c11-ae02-0c06fcf37940 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.072382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c6fb6-1d92-41fc-95ac-11588c121542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.073037 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6fb6-1d92-41fc-95ac-11588c121542 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.073317 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6fb6-1d92-41fc-95ac-11588c121542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.073585 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6fb6-1d92-41fc-95ac-11588c121542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.074325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c6fb6-1d92-41fc-95ac-11588c121542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.076121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c6fb6-1d92-41fc-95ac-11588c121542 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.087503 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1355/2681] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:34.091438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c462c0-8638-4e18-9436-74e328ec714c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.102839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c462c0-8638-4e18-9436-74e328ec714c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.108635 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe681066-e99f-4f61-8881-c647748ec387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.109981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681066-e99f-4f61-8881-c647748ec387 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.110264 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681066-e99f-4f61-8881-c647748ec387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.110539 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681066-e99f-4f61-8881-c647748ec387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.111312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681066-e99f-4f61-8881-c647748ec387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.112931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681066-e99f-4f61-8881-c647748ec387 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.124164 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1327/2682] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:34.128796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c6fb6-1d92-41fc-95ac-11588c121542 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.141478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c6fb6-1d92-41fc-95ac-11588c121542 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.146328 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b42d0-6850-4bbe-b20b-6cdd48976143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.147150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b42d0-6850-4bbe-b20b-6cdd48976143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.147514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b42d0-6850-4bbe-b20b-6cdd48976143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.147786 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b42d0-6850-4bbe-b20b-6cdd48976143 admin admin] PATH_INFO: `/v3/projects/1c7849588b714a1fa02035aae9978744/users/0d835f927ff14071b7a5ba2fd85cb644/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.163854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b42d0-6850-4bbe-b20b-6cdd48976143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7849588b714a1fa02035aae9978744, user_id=0d835f927ff14071b7a5ba2fd85cb644, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.166650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b42d0-6850-4bbe-b20b-6cdd48976143 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.181128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1356/2683] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:32:34 2022] PUT /identity/v3/projects/1c7849588b714a1fa02035aae9978744/users/0d835f927ff14071b7a5ba2fd85cb644/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:34.192829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b18bc3-7540-4420-af08-520c35a6d364 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.193390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b18bc3-7540-4420-af08-520c35a6d364 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.193390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b18bc3-7540-4420-af08-520c35a6d364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.222647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-49b18bc3-7540-4420-af08-520c35a6d364 None None] MFA Rules not processed for user `0d835f927ff14071b7a5ba2fd85cb644`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:34.223881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-49b18bc3-7540-4420-af08-520c35a6d364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:34.289434 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1328/2684] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:34.293795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b42d0-6850-4bbe-b20b-6cdd48976143 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.337280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:34.363569 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b42d0-6850-4bbe-b20b-6cdd48976143 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.369457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b62853-0779-48ac-ac90-ef2d2326e715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.370154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b62853-0779-48ac-ac90-ef2d2326e715 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.370431 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b62853-0779-48ac-ac90-ef2d2326e715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.370767 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b62853-0779-48ac-ac90-ef2d2326e715 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.371542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b62853-0779-48ac-ac90-ef2d2326e715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.373457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b62853-0779-48ac-ac90-ef2d2326e715 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.404737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1357/2685] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:32:34 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:32:34.408223 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b18bc3-7540-4420-af08-520c35a6d364 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.418805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b18bc3-7540-4420-af08-520c35a6d364 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.486868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9919868-edfc-4990-b9ed-16658a2b1af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.487581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9919868-edfc-4990-b9ed-16658a2b1af1 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.487862 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9919868-edfc-4990-b9ed-16658a2b1af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.488133 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9919868-edfc-4990-b9ed-16658a2b1af1 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.488857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9919868-edfc-4990-b9ed-16658a2b1af1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.491417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9919868-edfc-4990-b9ed-16658a2b1af1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.508345 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:34.518768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1329/2686] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:32:34 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:32:34.519844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b62853-0779-48ac-ac90-ef2d2326e715 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.531964 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b62853-0779-48ac-ac90-ef2d2326e715 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.536929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20289d9d-cd5e-429e-9bc4-2179a45236d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.537632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289d9d-cd5e-429e-9bc4-2179a45236d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.537917 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289d9d-cd5e-429e-9bc4-2179a45236d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.538191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289d9d-cd5e-429e-9bc4-2179a45236d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.538928 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20289d9d-cd5e-429e-9bc4-2179a45236d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.540630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20289d9d-cd5e-429e-9bc4-2179a45236d4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.551872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1358/2687] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:34.556054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9919868-edfc-4990-b9ed-16658a2b1af1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.566393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9919868-edfc-4990-b9ed-16658a2b1af1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.571031 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d913c2-5a86-4c69-ae50-756b9f5a882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.571961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d913c2-5a86-4c69-ae50-756b9f5a882c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.572242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d913c2-5a86-4c69-ae50-756b9f5a882c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.572516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d913c2-5a86-4c69-ae50-756b9f5a882c admin admin] PATH_INFO: `/v3/projects/be2b822243f245bc937ba7cfc280f377/users/231b1a581a2446289e4dc8d2a060807e/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.589573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d913c2-5a86-4c69-ae50-756b9f5a882c admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2b822243f245bc937ba7cfc280f377, user_id=231b1a581a2446289e4dc8d2a060807e, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.592581 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d913c2-5a86-4c69-ae50-756b9f5a882c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.611279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1330/2688] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:32:34 2022] PUT /identity/v3/projects/be2b822243f245bc937ba7cfc280f377/users/231b1a581a2446289e4dc8d2a060807e/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:34.614645 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20289d9d-cd5e-429e-9bc4-2179a45236d4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.631799 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20289d9d-cd5e-429e-9bc4-2179a45236d4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.690365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf78d51-23b9-46ee-b33e-123617395107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.690969 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf78d51-23b9-46ee-b33e-123617395107 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.691267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf78d51-23b9-46ee-b33e-123617395107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.691552 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf78d51-23b9-46ee-b33e-123617395107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.692279 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf78d51-23b9-46ee-b33e-123617395107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.693941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf78d51-23b9-46ee-b33e-123617395107 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.706396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1359/2689] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:34.708458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d913c2-5a86-4c69-ae50-756b9f5a882c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.719339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d913c2-5a86-4c69-ae50-756b9f5a882c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.724242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.725212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.725560 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.725903 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] PATH_INFO: `/v3/projects/be2b822243f245bc937ba7cfc280f377/users/231b1a581a2446289e4dc8d2a060807e/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.727974 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2b822243f245bc937ba7cfc280f377, user_id=231b1a581a2446289e4dc8d2a060807e, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.730683 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.741956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1331/2690] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:32:34 2022] PUT /identity/v3/projects/be2b822243f245bc937ba7cfc280f377/users/231b1a581a2446289e4dc8d2a060807e/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:32:34.752362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2fcd-64cf-48a1-a3e1-40a922d2f39b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.752710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2fcd-64cf-48a1-a3e1-40a922d2f39b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.752990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2fcd-64cf-48a1-a3e1-40a922d2f39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.780757 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d74d2fcd-64cf-48a1-a3e1-40a922d2f39b None None] MFA Rules not processed for user `231b1a581a2446289e4dc8d2a060807e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:34.781981 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d74d2fcd-64cf-48a1-a3e1-40a922d2f39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:34.857753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1360/2691] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:34.868963 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:34.870762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:34.929336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b789a0-2509-4735-b2e3-5aea8c15c0ca admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:34.933184 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01f036-e613-4292-83ec-3d575a9d4aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:34.933829 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f036-e613-4292-83ec-3d575a9d4aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:34.934108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f036-e613-4292-83ec-3d575a9d4aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:34.934378 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f036-e613-4292-83ec-3d575a9d4aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:34.944130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01f036-e613-4292-83ec-3d575a9d4aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:34.945507 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01f036-e613-4292-83ec-3d575a9d4aa8 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:34.955462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1332/2692] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:35.575274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d2fcd-64cf-48a1-a3e1-40a922d2f39b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:35.612868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:35.656038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d2fcd-64cf-48a1-a3e1-40a922d2f39b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:35.664222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:35.665325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:35.665701 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:35.666098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] PATH_INFO: `/v3/users/0d835f927ff14071b7a5ba2fd85cb644` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:35.667741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d835f927ff14071b7a5ba2fd85cb644)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:35.669842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:35.739716 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d835f927ff14071b7a5ba2fd85cb644'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:35.740391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d835f927ff14071b7a5ba2fd85cb644'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:35.750316 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d835f927ff14071b7a5ba2fd85cb644'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:35.767022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d835f927ff14071b7a5ba2fd85cb644'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:35.777312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d835f927ff14071b7a5ba2fd85cb644'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:35.790754 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1361/2693] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:32:35 2022] DELETE /identity/v3/users/0d835f927ff14071b7a5ba2fd85cb644 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:35.792234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e542b-ffe4-4e25-8023-c423102869b5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:35.792763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e542b-ffe4-4e25-8023-c423102869b5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:35.792763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e542b-ffe4-4e25-8023-c423102869b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:35.812718 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a60e542b-ffe4-4e25-8023-c423102869b5 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:35.813948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a60e542b-ffe4-4e25-8023-c423102869b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:35.888324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1333/2694] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:32:35.897049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:35.898566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:35.961172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b323d85-5bb8-4fca-8bcd-8d4c41669bb1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:35.965008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdc317-d823-4f5e-84e5-2ef04f9e1553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:35.965738 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdc317-d823-4f5e-84e5-2ef04f9e1553 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:35.966085 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdc317-d823-4f5e-84e5-2ef04f9e1553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:35.966419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdc317-d823-4f5e-84e5-2ef04f9e1553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:35.976087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdc317-d823-4f5e-84e5-2ef04f9e1553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:35.977588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdc317-d823-4f5e-84e5-2ef04f9e1553 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:35.988703 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1362/2695] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:36.292334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e542b-ffe4-4e25-8023-c423102869b5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:36.293803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a60e542b-ffe4-4e25-8023-c423102869b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:36.306208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e542b-ffe4-4e25-8023-c423102869b5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:36.310790 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:36.311780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:36.312415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:36.312524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] PATH_INFO: `/v3/projects/1c7849588b714a1fa02035aae9978744` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:36.313636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7849588b714a1fa02035aae9978744)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:36.315656 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:36.367076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7849588b714a1fa02035aae9978744', 'request_id': 'req-be346427-bae0-4c92-82e3-ecbb1a78a578'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.373801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7849588b714a1fa02035aae9978744', 'request_id': 'req-be346427-bae0-4c92-82e3-ecbb1a78a578'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.375757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:36.386131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7849588b714a1fa02035aae9978744', 'request_id': 'req-be346427-bae0-4c92-82e3-ecbb1a78a578'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.404357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] The token cache is being invalidate because project 1c7849588b714a1fa02035aae9978744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:32:36.404763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.405615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.407245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1334/2696] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:32:36 2022] DELETE /identity/v3/projects/1c7849588b714a1fa02035aae9978744 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:36.411385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdc317-d823-4f5e-84e5-2ef04f9e1553 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:36.421371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdc317-d823-4f5e-84e5-2ef04f9e1553 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:36.470064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:36.480607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:36.481532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:36.482064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:36.482586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] PATH_INFO: `/v3/users/231b1a581a2446289e4dc8d2a060807e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:36.484382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] RBAC: Authorizing `identity:delete_user(user_id=231b1a581a2446289e4dc8d2a060807e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:36.486389 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:36.537963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '231b1a581a2446289e4dc8d2a060807e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.538557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '231b1a581a2446289e4dc8d2a060807e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.545306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '231b1a581a2446289e4dc8d2a060807e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.557690 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '231b1a581a2446289e4dc8d2a060807e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.567182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-381b000b-8126-4120-8d76-0a99c899526c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '231b1a581a2446289e4dc8d2a060807e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:36.573749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1363/2697] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:32:36 2022] DELETE /identity/v3/users/231b1a581a2446289e4dc8d2a060807e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:36.867122 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:36.868807 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:36.884047 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be346427-bae0-4c92-82e3-ecbb1a78a578 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:36.947400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:36.948284 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:36.948580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:36.948850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] PATH_INFO: `/v3/projects/be2b822243f245bc937ba7cfc280f377` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:36.950026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2b822243f245bc937ba7cfc280f377)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:36.951968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.001290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2b822243f245bc937ba7cfc280f377', 'request_id': 'req-56032790-7cef-41a5-b66e-fd83300c0e46'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.006941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2b822243f245bc937ba7cfc280f377', 'request_id': 'req-56032790-7cef-41a5-b66e-fd83300c0e46'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.016730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2b822243f245bc937ba7cfc280f377', 'request_id': 'req-56032790-7cef-41a5-b66e-fd83300c0e46'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.028002 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] The token cache is being invalidate because project be2b822243f245bc937ba7cfc280f377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:32:37.028314 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.028968 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56032790-7cef-41a5-b66e-fd83300c0e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.030559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1335/2698] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:32:36 2022] DELETE /identity/v3/projects/be2b822243f245bc937ba7cfc280f377 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:37.035952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742f25-69c7-4b12-9211-4e7114213b64 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.036454 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742f25-69c7-4b12-9211-4e7114213b64 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.036571 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742f25-69c7-4b12-9211-4e7114213b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.054998 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bd742f25-69c7-4b12-9211-4e7114213b64 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:37.056204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bd742f25-69c7-4b12-9211-4e7114213b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:37.125195 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1364/2699] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:37.126928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559f2d6-5923-4777-90c5-02081cf7761a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.128006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559f2d6-5923-4777-90c5-02081cf7761a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.128451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559f2d6-5923-4777-90c5-02081cf7761a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.149527 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5559f2d6-5923-4777-90c5-02081cf7761a None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:37.150769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5559f2d6-5923-4777-90c5-02081cf7761a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:37.160155 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1336/2700] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:37.164427 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742f25-69c7-4b12-9211-4e7114213b64 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:37.175638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742f25-69c7-4b12-9211-4e7114213b64 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:37.180556 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:37.181262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.181574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.181854 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.182593 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:32:37.182904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:37.184578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.197312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1365/2701] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:32:37.201713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559f2d6-5923-4777-90c5-02081cf7761a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:37.211436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559f2d6-5923-4777-90c5-02081cf7761a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:37.216577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965c4bd-ea8a-442c-94c6-13d02da88be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:37.217238 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965c4bd-ea8a-442c-94c6-13d02da88be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.217518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965c4bd-ea8a-442c-94c6-13d02da88be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.217788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965c4bd-ea8a-442c-94c6-13d02da88be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.218480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965c4bd-ea8a-442c-94c6-13d02da88be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:37.220728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965c4bd-ea8a-442c-94c6-13d02da88be5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.248463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1337/2702] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:32:37 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:32:37.253080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:37.266934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13274f-7e66-4849-b6e8-178c33f0cc16 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:37.327505 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f1f89-e208-4b2e-861c-e5f64bb18baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:37.328092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1f89-e208-4b2e-861c-e5f64bb18baa admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.328366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1f89-e208-4b2e-861c-e5f64bb18baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.328637 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1f89-e208-4b2e-861c-e5f64bb18baa admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.329346 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f1f89-e208-4b2e-861c-e5f64bb18baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:37.331321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f1f89-e208-4b2e-861c-e5f64bb18baa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.338449 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:37.354456 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1366/2703] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:32:37 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:32:37.358640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965c4bd-ea8a-442c-94c6-13d02da88be5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:37.369991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965c4bd-ea8a-442c-94c6-13d02da88be5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:37.375371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31699434-4c64-4c12-a7be-0ae7cd2bdf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:37.376327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31699434-4c64-4c12-a7be-0ae7cd2bdf34 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.376712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31699434-4c64-4c12-a7be-0ae7cd2bdf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.376995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31699434-4c64-4c12-a7be-0ae7cd2bdf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.377726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31699434-4c64-4c12-a7be-0ae7cd2bdf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:37.379383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31699434-4c64-4c12-a7be-0ae7cd2bdf34 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.390195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1338/2704] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:37.394869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f1f89-e208-4b2e-861c-e5f64bb18baa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:37.409150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f1f89-e208-4b2e-861c-e5f64bb18baa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:37.414759 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eabcfb-a258-499b-9a44-e94328ac79af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:37.415459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eabcfb-a258-499b-9a44-e94328ac79af admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.415727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eabcfb-a258-499b-9a44-e94328ac79af admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.415993 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eabcfb-a258-499b-9a44-e94328ac79af admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.416851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eabcfb-a258-499b-9a44-e94328ac79af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:37.418460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eabcfb-a258-499b-9a44-e94328ac79af admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.429091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1367/2705] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:37.433713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31699434-4c64-4c12-a7be-0ae7cd2bdf34 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:37.445088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31699434-4c64-4c12-a7be-0ae7cd2bdf34 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:37.453053 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:37.454385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.454673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.454948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] PATH_INFO: `/v3/projects/69323e4ffec84d51b6231d7ec42e54ec/users/20a3a90b62764759bcfff7a0e3a276d0/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.472333 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69323e4ffec84d51b6231d7ec42e54ec, user_id=20a3a90b62764759bcfff7a0e3a276d0, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:37.474971 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.493159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1339/2706] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:32:37 2022] PUT /identity/v3/projects/69323e4ffec84d51b6231d7ec42e54ec/users/20a3a90b62764759bcfff7a0e3a276d0/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:32:37.504648 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619033-1c65-4452-8de6-d96a78ae463f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.504938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619033-1c65-4452-8de6-d96a78ae463f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.505215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619033-1c65-4452-8de6-d96a78ae463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.532095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bc619033-1c65-4452-8de6-d96a78ae463f None None] MFA Rules not processed for user `20a3a90b62764759bcfff7a0e3a276d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:37.533529 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bc619033-1c65-4452-8de6-d96a78ae463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:37.605879 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1368/2707] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:37.618880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:37.621530 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:37.644085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:37.691628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc823d46-6aa6-4c36-bd7a-e2a047a0edf3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:37.696454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9a68d-2356-478b-96a4-038ff9bac948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:37.697109 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a68d-2356-478b-96a4-038ff9bac948 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.697392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a68d-2356-478b-96a4-038ff9bac948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.697664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a68d-2356-478b-96a4-038ff9bac948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.715541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9a68d-2356-478b-96a4-038ff9bac948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:37.717019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9a68d-2356-478b-96a4-038ff9bac948 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.729672 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1340/2708] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:37.753834 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc619033-1c65-4452-8de6-d96a78ae463f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:37.822112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc619033-1c65-4452-8de6-d96a78ae463f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:37.827494 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:37.828590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.828876 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.829150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] PATH_INFO: `/v3/users/20a3a90b62764759bcfff7a0e3a276d0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.830606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a3a90b62764759bcfff7a0e3a276d0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:37.832613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:37.890907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a3a90b62764759bcfff7a0e3a276d0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.892007 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a3a90b62764759bcfff7a0e3a276d0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.902837 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a3a90b62764759bcfff7a0e3a276d0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.915727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:37.918299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a3a90b62764759bcfff7a0e3a276d0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.928312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a3a90b62764759bcfff7a0e3a276d0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:37.935073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1369/2709] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:32:37 2022] DELETE /identity/v3/users/20a3a90b62764759bcfff7a0e3a276d0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:37.940213 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36d289-2650-495d-8f93-e7c90a6a6ebb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:37.940744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36d289-2650-495d-8f93-e7c90a6a6ebb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:37.940744 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36d289-2650-495d-8f93-e7c90a6a6ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:37.959613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0c36d289-2650-495d-8f93-e7c90a6a6ebb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:37.960883 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0c36d289-2650-495d-8f93-e7c90a6a6ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:38.024079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1341/2710] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:38.034476 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:38.036081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:38.098918 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7dd7d-5369-472f-8a65-1768a021d568 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:38.102882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff9291-0ae3-4dd4-aef8-7ee0757b3df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:38.103554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff9291-0ae3-4dd4-aef8-7ee0757b3df9 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:38.103831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff9291-0ae3-4dd4-aef8-7ee0757b3df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:38.104102 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff9291-0ae3-4dd4-aef8-7ee0757b3df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:38.113612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff9291-0ae3-4dd4-aef8-7ee0757b3df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:38.115055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff9291-0ae3-4dd4-aef8-7ee0757b3df9 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:38.124883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1370/2711] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:38.638779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36d289-2650-495d-8f93-e7c90a6a6ebb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:38.640297 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0c36d289-2650-495d-8f93-e7c90a6a6ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:38.654069 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36d289-2650-495d-8f93-e7c90a6a6ebb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:38.659506 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:38.660444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:38.660724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:38.660991 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] PATH_INFO: `/v3/projects/69323e4ffec84d51b6231d7ec42e54ec` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:38.662231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] RBAC: Authorizing `identity:delete_project(project_id=69323e4ffec84d51b6231d7ec42e54ec)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:38.664186 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:38.717805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69323e4ffec84d51b6231d7ec42e54ec', 'request_id': 'req-13b6cb3b-98ca-46c0-8fde-f47298b9febc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:38.724445 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69323e4ffec84d51b6231d7ec42e54ec', 'request_id': 'req-13b6cb3b-98ca-46c0-8fde-f47298b9febc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:38.733726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69323e4ffec84d51b6231d7ec42e54ec', 'request_id': 'req-13b6cb3b-98ca-46c0-8fde-f47298b9febc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:38.745263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] The token cache is being invalidate because project 69323e4ffec84d51b6231d7ec42e54ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:32:38.745565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:38.746231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13b6cb3b-98ca-46c0-8fde-f47298b9febc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:32:38.747848 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1342/2712] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:32:38 2022] DELETE /identity/v3/projects/69323e4ffec84d51b6231d7ec42e54ec => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:38.752916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad22a1-ccb5-426f-aed9-5c461f3073e1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:38.753203 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad22a1-ccb5-426f-aed9-5c461f3073e1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:38.753470 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad22a1-ccb5-426f-aed9-5c461f3073e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:38.771905 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f6ad22a1-ccb5-426f-aed9-5c461f3073e1 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:38.773206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f6ad22a1-ccb5-426f-aed9-5c461f3073e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:38.836104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1371/2713] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:38.845252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b75bc-852b-41b8-8269-38b487555178 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:38.845589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b75bc-852b-41b8-8269-38b487555178 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:38.845873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b75bc-852b-41b8-8269-38b487555178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:38.867449 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ef4b75bc-852b-41b8-8269-38b487555178 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:38.868696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ef4b75bc-852b-41b8-8269-38b487555178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:38.878173 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1343/2714] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:38.884951 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad22a1-ccb5-426f-aed9-5c461f3073e1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:38.900222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad22a1-ccb5-426f-aed9-5c461f3073e1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:38.906617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:38.907299 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:38.907675 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:38.908002 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:38.908686 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:32:38.908990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:38.910602 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:38.922379 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1372/2715] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:32:38.926677 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b75bc-852b-41b8-8269-38b487555178 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:38.936101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b75bc-852b-41b8-8269-38b487555178 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:38.941621 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5818170-c628-4181-8b59-a7164b2e8a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:38.942263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5818170-c628-4181-8b59-a7164b2e8a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:38.942509 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5818170-c628-4181-8b59-a7164b2e8a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:38.942809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5818170-c628-4181-8b59-a7164b2e8a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:38.943533 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5818170-c628-4181-8b59-a7164b2e8a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:38.945422 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5818170-c628-4181-8b59-a7164b2e8a17 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:38.975487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1344/2716] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:32:38 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:32:38.980872 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:38.992031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7aba-fe4d-4fb2-b5d7-943d0fbc7675 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:39.055144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0295c-5869-43f7-a1c3-cc79a0b28102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:39.055902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0295c-5869-43f7-a1c3-cc79a0b28102 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:39.056179 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0295c-5869-43f7-a1c3-cc79a0b28102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:39.056455 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0295c-5869-43f7-a1c3-cc79a0b28102 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:39.057157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0295c-5869-43f7-a1c3-cc79a0b28102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:39.059123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0295c-5869-43f7-a1c3-cc79a0b28102 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:39.082011 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1373/2717] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:32:38 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:32:39.086079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5818170-c628-4181-8b59-a7164b2e8a17 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:39.097064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5818170-c628-4181-8b59-a7164b2e8a17 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:39.102791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f5bb5-ce08-4515-b86f-2090e2c04d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:39.103493 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f5bb5-ce08-4515-b86f-2090e2c04d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:39.103774 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f5bb5-ce08-4515-b86f-2090e2c04d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:39.104041 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f5bb5-ce08-4515-b86f-2090e2c04d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:39.104776 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f5bb5-ce08-4515-b86f-2090e2c04d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:39.106438 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f5bb5-ce08-4515-b86f-2090e2c04d57 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:39.113486 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:39.120410 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1345/2718] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:32:39.123240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0295c-5869-43f7-a1c3-cc79a0b28102 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:39.134332 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0295c-5869-43f7-a1c3-cc79a0b28102 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:39.141766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d3979-80e3-4bdf-93a6-47fa9971842a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:39.142914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d3979-80e3-4bdf-93a6-47fa9971842a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:39.143298 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d3979-80e3-4bdf-93a6-47fa9971842a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:39.143671 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d3979-80e3-4bdf-93a6-47fa9971842a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:39.144788 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d3979-80e3-4bdf-93a6-47fa9971842a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:39.147337 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d3979-80e3-4bdf-93a6-47fa9971842a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:39.149291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:39.167249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1374/2719] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:32:39.171342 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f5bb5-ce08-4515-b86f-2090e2c04d57 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:39.181851 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f5bb5-ce08-4515-b86f-2090e2c04d57 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:39.188073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:39.189008 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:39.189299 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:39.189703 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] PATH_INFO: `/v3/projects/58375218b42f46abb19fb2772c05fd1d/users/a07de9cf87b241e4948eaa4939b70289/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:39.205168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58375218b42f46abb19fb2772c05fd1d, user_id=a07de9cf87b241e4948eaa4939b70289, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:39.207794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:39.227352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1346/2720] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:32:39 2022] PUT /identity/v3/projects/58375218b42f46abb19fb2772c05fd1d/users/a07de9cf87b241e4948eaa4939b70289/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:32:39.237940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc03ed-7475-422d-b805-b4b14e1a6787 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:39.238251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc03ed-7475-422d-b805-b4b14e1a6787 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:39.238514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc03ed-7475-422d-b805-b4b14e1a6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:39.265985 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b5fc03ed-7475-422d-b805-b4b14e1a6787 None None] MFA Rules not processed for user `a07de9cf87b241e4948eaa4939b70289`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:32:39.267501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5fc03ed-7475-422d-b805-b4b14e1a6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:39.335713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1375/2721] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:32:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:32:39.347975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:39.349948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:39.411962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99621bf-9b50-415b-b995-a3a646d3c048 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:39.416391 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8cbcc9295074710aeacb35e4e1a4348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:39.417050 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:39.417334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:39.417667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:39.426488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:39.428024 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:39.436793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1347/2722] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:46.911434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc03ed-7475-422d-b805-b4b14e1a6787 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:46.913263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5fc03ed-7475-422d-b805-b4b14e1a6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:46.979784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc03ed-7475-422d-b805-b4b14e1a6787 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:46.984262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:46.984916 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:46.985193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:46.985462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:46.987615 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:47.058176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:47.060081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:47.070568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1376/2723] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:48.734926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:32:48.736685 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:48.818432 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a6546-92e3-4c83-9468-19ed97584d0a service glance] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:32:48.825293 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d39d-44f3-4a28-b8de-dc664ed8a5a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:32:48.825993 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d39d-44f3-4a28-b8de-dc664ed8a5a2 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:32:48.826279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d39d-44f3-4a28-b8de-dc664ed8a5a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:32:48.826546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d39d-44f3-4a28-b8de-dc664ed8a5a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:32:48.828577 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5619d39d-44f3-4a28-b8de-dc664ed8a5a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:32:48.842787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619d39d-44f3-4a28-b8de-dc664ed8a5a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:32:48.844452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619d39d-44f3-4a28-b8de-dc664ed8a5a2 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:32:48.857306 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1348/2724] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:32:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:32:57.066172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:32:58.851845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=70,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:33:04.576833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:04.650539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef38e49-2292-4ca4-9370-536406b39f1d service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:04.657098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:04.658044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:04.658327 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:04.658599 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] PATH_INFO: `/v3/users/a07de9cf87b241e4948eaa4939b70289` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:04.659934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07de9cf87b241e4948eaa4939b70289)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:04.661877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:04.721889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07de9cf87b241e4948eaa4939b70289'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:04.721889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07de9cf87b241e4948eaa4939b70289'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:04.733353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07de9cf87b241e4948eaa4939b70289'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:04.752457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07de9cf87b241e4948eaa4939b70289'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:04.766894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07de9cf87b241e4948eaa4939b70289'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:04.779068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1377/2725] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:33:04 2022] DELETE /identity/v3/users/a07de9cf87b241e4948eaa4939b70289 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:33:04.785152 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a4e2f-0582-4521-aa08-709879dc26dd None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:04.785849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a4e2f-0582-4521-aa08-709879dc26dd None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:04.785849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a4e2f-0582-4521-aa08-709879dc26dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:04.815495 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3b0a4e2f-0582-4521-aa08-709879dc26dd None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:33:04.816855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3b0a4e2f-0582-4521-aa08-709879dc26dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:04.895760 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1349/2726] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:33:04.909088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:04.911947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:04.986727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464deb8-4211-42c9-b93e-9d0eb2ee108d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:04.991959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460c091-6c51-4a49-a180-a7b842b723ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:04.992639 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c091-6c51-4a49-a180-a7b842b723ce service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:04.992924 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c091-6c51-4a49-a180-a7b842b723ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:04.993158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c091-6c51-4a49-a180-a7b842b723ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.004043 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460c091-6c51-4a49-a180-a7b842b723ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:05.005498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460c091-6c51-4a49-a180-a7b842b723ce service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:05.017374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1378/2727] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:33:05.338629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a4e2f-0582-4521-aa08-709879dc26dd None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:05.341013 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3b0a4e2f-0582-4521-aa08-709879dc26dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:05.362324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a4e2f-0582-4521-aa08-709879dc26dd None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:05.370512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:05.371795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.372127 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.372496 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] PATH_INFO: `/v3/projects/58375218b42f46abb19fb2772c05fd1d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.374583 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58375218b42f46abb19fb2772c05fd1d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:05.377430 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:05.437520 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58375218b42f46abb19fb2772c05fd1d', 'request_id': 'req-cb2843fc-8439-408c-b9c2-69fc9455e21d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:05.444161 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58375218b42f46abb19fb2772c05fd1d', 'request_id': 'req-cb2843fc-8439-408c-b9c2-69fc9455e21d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:05.458082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58375218b42f46abb19fb2772c05fd1d', 'request_id': 'req-cb2843fc-8439-408c-b9c2-69fc9455e21d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:05.472813 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] The token cache is being invalidate because project 58375218b42f46abb19fb2772c05fd1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:33:05.473103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:05.473941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-cb2843fc-8439-408c-b9c2-69fc9455e21d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:33:05.476026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1350/2728] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:33:05 2022] DELETE /identity/v3/projects/58375218b42f46abb19fb2772c05fd1d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:33:05.487016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c16cb-7829-4e72-af85-192adeef8b3d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.487357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c16cb-7829-4e72-af85-192adeef8b3d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.487601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c16cb-7829-4e72-af85-192adeef8b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.507184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b12c16cb-7829-4e72-af85-192adeef8b3d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:33:05.508452 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b12c16cb-7829-4e72-af85-192adeef8b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:05.533362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:33:05.579087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1379/2729] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:33:05.586978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8e625-3e86-417f-ad4e-3f8fe672e8e3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.587266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8e625-3e86-417f-ad4e-3f8fe672e8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.587580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8e625-3e86-417f-ad4e-3f8fe672e8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.606842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-30d8e625-3e86-417f-ad4e-3f8fe672e8e3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:33:05.608085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-30d8e625-3e86-417f-ad4e-3f8fe672e8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:05.617818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1351/2730] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:33:05.622240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c16cb-7829-4e72-af85-192adeef8b3d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:05.632973 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c16cb-7829-4e72-af85-192adeef8b3d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:05.638564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:05.639232 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.639535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.639889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.640676 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:33:05.640972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:05.642581 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:05.655365 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1380/2731] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:33:05.659935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8e625-3e86-417f-ad4e-3f8fe672e8e3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:05.670111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8e625-3e86-417f-ad4e-3f8fe672e8e3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:05.676124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ef871-7387-4236-84cb-ddd14044e9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:05.676791 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef871-7387-4236-84cb-ddd14044e9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.677074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef871-7387-4236-84cb-ddd14044e9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.677369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef871-7387-4236-84cb-ddd14044e9f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.678068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ef871-7387-4236-84cb-ddd14044e9f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:05.680023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ef871-7387-4236-84cb-ddd14044e9f2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:05.707420 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1352/2732] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:33:05 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:33:05.713761 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:05.728814 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee85d0-dda6-4cbc-b6b3-63aaac1736e2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:05.795745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6474a42-8bb7-4a5e-9edd-918abf3f2e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:05.796542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6474a42-8bb7-4a5e-9edd-918abf3f2e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.796826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6474a42-8bb7-4a5e-9edd-918abf3f2e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.797087 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6474a42-8bb7-4a5e-9edd-918abf3f2e13 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.797793 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6474a42-8bb7-4a5e-9edd-918abf3f2e13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:05.799749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6474a42-8bb7-4a5e-9edd-918abf3f2e13 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:05.826681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1381/2733] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:33:05 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:33:05.832119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ef871-7387-4236-84cb-ddd14044e9f2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:05.848241 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ef871-7387-4236-84cb-ddd14044e9f2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:05.855640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71361c-d818-44f3-b788-d8bde834dba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:05.856305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71361c-d818-44f3-b788-d8bde834dba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.856590 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71361c-d818-44f3-b788-d8bde834dba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.856861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71361c-d818-44f3-b788-d8bde834dba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.857627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71361c-d818-44f3-b788-d8bde834dba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:05.859896 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71361c-d818-44f3-b788-d8bde834dba0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:05.871812 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1353/2734] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:33:05.876623 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6474a42-8bb7-4a5e-9edd-918abf3f2e13 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:05.889803 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6474a42-8bb7-4a5e-9edd-918abf3f2e13 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:05.895338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e339775-3e55-425a-89d3-b87cd3b9d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:05.896031 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e339775-3e55-425a-89d3-b87cd3b9d307 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.896308 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e339775-3e55-425a-89d3-b87cd3b9d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.896582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e339775-3e55-425a-89d3-b87cd3b9d307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.897307 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e339775-3e55-425a-89d3-b87cd3b9d307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:05.898944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e339775-3e55-425a-89d3-b87cd3b9d307 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:05.909539 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1382/2735] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:33:05.914076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71361c-d818-44f3-b788-d8bde834dba0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:05.925390 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71361c-d818-44f3-b788-d8bde834dba0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:05.931146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:05.932088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.932371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.932647 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] PATH_INFO: `/v3/projects/86a447448dde4ecd80717993f58f33a7/users/1eb79dd547c04d2eac6dc1e35311bd9c/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.948371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] RBAC: Authorizing `identity:create_grant(project_id=86a447448dde4ecd80717993f58f33a7, user_id=1eb79dd547c04d2eac6dc1e35311bd9c, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:05.951026 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:05.965436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1354/2736] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:33:05 2022] PUT /identity/v3/projects/86a447448dde4ecd80717993f58f33a7/users/1eb79dd547c04d2eac6dc1e35311bd9c/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:33:05.971583 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fb39c-1f9b-4118-bb23-162d579c6630 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:05.971897 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fb39c-1f9b-4118-bb23-162d579c6630 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:05.972306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fb39c-1f9b-4118-bb23-162d579c6630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:05.993947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-486fb39c-1f9b-4118-bb23-162d579c6630 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:33:05.995130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-486fb39c-1f9b-4118-bb23-162d579c6630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:06.066784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1383/2737] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:33:06.076563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:06.078855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:06.103194 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:33:06.140550 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eede5cf-00d4-41f7-a171-654f9f5166af admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:06.144730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d0b3c-8d6e-440d-97a0-a943390faa42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:06.145376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b3c-8d6e-440d-97a0-a943390faa42 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:06.145655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b3c-8d6e-440d-97a0-a943390faa42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:06.145925 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b3c-8d6e-440d-97a0-a943390faa42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:06.155949 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d0b3c-8d6e-440d-97a0-a943390faa42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:06.157400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d0b3c-8d6e-440d-97a0-a943390faa42 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:06.167139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1355/2738] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:33:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:33:10.068362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71395e0a-9394-43a1-aab2-74728558326a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:10.068862 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71395e0a-9394-43a1-aab2-74728558326a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:10.068967 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71395e0a-9394-43a1-aab2-74728558326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:10.106607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-71395e0a-9394-43a1-aab2-74728558326a None None] MFA Rules not processed for user `1eb79dd547c04d2eac6dc1e35311bd9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:33:10.107847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-71395e0a-9394-43a1-aab2-74728558326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:10.132824 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:33:10.183961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1384/2739] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:33:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:33:10.188700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d0b3c-8d6e-440d-97a0-a943390faa42 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:10.198675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d0b3c-8d6e-440d-97a0-a943390faa42 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:10.203728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cb5ce-3a5a-4b88-a490-ab63f61591d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:10.204389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb5ce-3a5a-4b88-a490-ab63f61591d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:10.204669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb5ce-3a5a-4b88-a490-ab63f61591d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:10.204941 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb5ce-3a5a-4b88-a490-ab63f61591d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:10.205673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cb5ce-3a5a-4b88-a490-ab63f61591d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:10.207659 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cb5ce-3a5a-4b88-a490-ab63f61591d2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:10.244679 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1356/2740] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:33:10 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:33:10.257892 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71395e0a-9394-43a1-aab2-74728558326a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:10.273875 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71395e0a-9394-43a1-aab2-74728558326a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:10.345531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4928653-9c55-425c-abe0-d9cfb0687051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:10.346305 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4928653-9c55-425c-abe0-d9cfb0687051 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:10.346657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4928653-9c55-425c-abe0-d9cfb0687051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:10.346988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4928653-9c55-425c-abe0-d9cfb0687051 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:10.347877 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4928653-9c55-425c-abe0-d9cfb0687051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:10.350206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4928653-9c55-425c-abe0-d9cfb0687051 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:10.376853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1385/2741] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:33:10 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:33:10.382696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cb5ce-3a5a-4b88-a490-ab63f61591d2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:10.400981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cb5ce-3a5a-4b88-a490-ab63f61591d2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:10.405994 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe6ce1-62c0-4c9d-a84d-bd82c06e39f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:10.406691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe6ce1-62c0-4c9d-a84d-bd82c06e39f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:10.406975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe6ce1-62c0-4c9d-a84d-bd82c06e39f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:10.407279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe6ce1-62c0-4c9d-a84d-bd82c06e39f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:10.408081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe6ce1-62c0-4c9d-a84d-bd82c06e39f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:10.409762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe6ce1-62c0-4c9d-a84d-bd82c06e39f8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:10.426068 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1357/2742] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:33:10.430112 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4928653-9c55-425c-abe0-d9cfb0687051 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:10.442780 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4928653-9c55-425c-abe0-d9cfb0687051 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:10.447756 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7540c3a-f60a-4397-a445-789a15a60051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:10.448678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7540c3a-f60a-4397-a445-789a15a60051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:10.448956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7540c3a-f60a-4397-a445-789a15a60051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:10.449230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7540c3a-f60a-4397-a445-789a15a60051 admin admin] PATH_INFO: `/v3/projects/102891aa9e4c42e589476bd2b684948b/users/6d07a2cec61641408a74413b57711149/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:10.465881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7540c3a-f60a-4397-a445-789a15a60051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102891aa9e4c42e589476bd2b684948b, user_id=6d07a2cec61641408a74413b57711149, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:10.468941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7540c3a-f60a-4397-a445-789a15a60051 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:10.484797 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1386/2743] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:33:10 2022] PUT /identity/v3/projects/102891aa9e4c42e589476bd2b684948b/users/6d07a2cec61641408a74413b57711149/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:33:10.488559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe6ce1-62c0-4c9d-a84d-bd82c06e39f8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:10.499770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe6ce1-62c0-4c9d-a84d-bd82c06e39f8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:10.561640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e158b3-37d6-4e47-89da-c8333905a26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:10.562417 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e158b3-37d6-4e47-89da-c8333905a26f admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:10.562743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e158b3-37d6-4e47-89da-c8333905a26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:10.563023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e158b3-37d6-4e47-89da-c8333905a26f admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:10.563816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e158b3-37d6-4e47-89da-c8333905a26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:10.565488 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e158b3-37d6-4e47-89da-c8333905a26f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:10.579962 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1358/2744] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:33:10.581234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7540c3a-f60a-4397-a445-789a15a60051 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:10.592623 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7540c3a-f60a-4397-a445-789a15a60051 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:10.597631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:10.598653 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:10.599010 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:10.599395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] PATH_INFO: `/v3/projects/102891aa9e4c42e589476bd2b684948b/users/6d07a2cec61641408a74413b57711149/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:10.601541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102891aa9e4c42e589476bd2b684948b, user_id=6d07a2cec61641408a74413b57711149, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:10.604632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:10.616100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1387/2745] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:33:10 2022] PUT /identity/v3/projects/102891aa9e4c42e589476bd2b684948b/users/6d07a2cec61641408a74413b57711149/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:33:14.918111 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980beb-e971-4459-acbf-215b57699502 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:14.918674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980beb-e971-4459-acbf-215b57699502 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:14.918674 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980beb-e971-4459-acbf-215b57699502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:14.945176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1d980beb-e971-4459-acbf-215b57699502 None None] MFA Rules not processed for user `6d07a2cec61641408a74413b57711149`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:33:14.946273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1d980beb-e971-4459-acbf-215b57699502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:15.019001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1359/2746] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:33:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:33:15.035070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:15.037291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:15.115469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3dd5a-998a-4aa5-8b24-4bc8b9c39213 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:15.121486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:15.122138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:15.122420 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:15.122692 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:15.128546 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:33:15.185437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:15.187359 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:15.201905 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1388/2747] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:33:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:33:16.250204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135509b1-1a12-45f9-9340-a29eb3ae16b6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:16.250786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135509b1-1a12-45f9-9340-a29eb3ae16b6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:16.250786 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135509b1-1a12-45f9-9340-a29eb3ae16b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:16.268599 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-135509b1-1a12-45f9-9340-a29eb3ae16b6 None None] MFA Rules not processed for user `1eb79dd547c04d2eac6dc1e35311bd9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:33:16.269795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-135509b1-1a12-45f9-9340-a29eb3ae16b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:16.276756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1360/2748] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:33:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:33:25.192719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=24 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:33:26.263167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:33:30.776151 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:30.777752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:30.929548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5f422-9b0e-4182-b04c-0342eba64425 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:30.936784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:30.937444 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:30.937722 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:30.938001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:30.940215 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:33:30.953641 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:30.955240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:30.966194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1389/2749] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:33:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:33:40.961465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=63 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:33:55.478878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135509b1-1a12-45f9-9340-a29eb3ae16b6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:33:55.560407 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135509b1-1a12-45f9-9340-a29eb3ae16b6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:33:55.564622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:33:55.565263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:33:55.565540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:33:55.565810 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:33:55.575602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:33:55.577038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:33:55.586648 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1361/2750] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:33:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:34:05.582476 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=32 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:35:20.058345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:20.068618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a175cb7-a95f-440a-9782-94c851e43f80 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:20.072204 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:20.072849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:20.073122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:20.073537 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:20.075435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:20.109418 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:20.111001 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:20.132263 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1390/2751] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:35:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:35:21.310860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:21.312471 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:21.339963 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b914031-9084-4f8c-9fa2-c5dda75234ef service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:21.343997 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:21.344646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:21.344921 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:21.345189 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:21.347150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:21.362019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:21.363510 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:21.375375 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1362/2752] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:35:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:35:30.129524 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=18 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:35:31.370130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=11 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:35:31.502440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:31.512536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933f8c9-b5bd-4e40-8c86-2c44a4dce0c2 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:31.516740 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:31.517457 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:31.517735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:31.518008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:31.519950 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:31.603831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:31.605320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:31.625831 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1391/2753] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:35:34.481186 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:34.521300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e154c-9e29-4725-9661-b6a2d34009f7 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:34.526156 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:34.527061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:34.527540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:34.527990 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:34.530529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:34.545278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:34.546092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:34.561337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1363/2754] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:35:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:35:38.021946 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:38.023933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:38.036472 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6df73-7403-4dac-8929-f763b5066ac4 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:38.041419 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:38.042319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:38.042601 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:38.042878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] PATH_INFO: `/v3/users/8d661ea344d74681bb7206f203033c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:38.044126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d661ea344d74681bb7206f203033c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:38.046100 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:38.101093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d661ea344d74681bb7206f203033c60'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.101750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d661ea344d74681bb7206f203033c60'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.109104 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d661ea344d74681bb7206f203033c60'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.121822 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d661ea344d74681bb7206f203033c60'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.131167 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d661ea344d74681bb7206f203033c60'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.137682 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1392/2755] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:35:38 2022] DELETE /identity/v3/users/8d661ea344d74681bb7206f203033c60 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:35:38.310716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:38.328055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8adc-04ea-4d30-86f1-8be66a215831 service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:38.393179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:38.394096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:38.394380 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:38.394726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] PATH_INFO: `/v3/projects/aa6a422103704f6d982c2bc558b3387e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:38.396085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa6a422103704f6d982c2bc558b3387e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:38.398011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:38.446773 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa6a422103704f6d982c2bc558b3387e', 'request_id': 'req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.452243 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa6a422103704f6d982c2bc558b3387e', 'request_id': 'req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.460669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa6a422103704f6d982c2bc558b3387e', 'request_id': 'req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.472169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] The token cache is being invalidate because project aa6a422103704f6d982c2bc558b3387e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:35:38.472489 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.473139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.475273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1364/2756] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:35:38 2022] DELETE /identity/v3/projects/aa6a422103704f6d982c2bc558b3387e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:35:38.479402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:38.481062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:38.498289 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f09d0-00d7-4ee6-a1f4-dc17047c0fa2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:38.533045 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:35:38.558733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:38.559655 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:38.559936 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:38.560206 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] PATH_INFO: `/v3/users/6c2aed93dad44ede9683370430c152a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:38.561405 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2aed93dad44ede9683370430c152a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:38.563387 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:38.615034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2aed93dad44ede9683370430c152a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.615753 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2aed93dad44ede9683370430c152a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.622975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2aed93dad44ede9683370430c152a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.636036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2aed93dad44ede9683370430c152a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.646236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2aed93dad44ede9683370430c152a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.652371 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1393/2757] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:35:38 2022] DELETE /identity/v3/users/6c2aed93dad44ede9683370430c152a5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:35:38.826573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:38.828219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:38.841967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250a6c-e49d-46eb-b7d7-7daf2c35aba6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:38.906726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:38.907563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:38.907845 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:38.908116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] PATH_INFO: `/v3/projects/598ad12efb0a4e5ab6a14de501494214` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:38.909308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=598ad12efb0a4e5ab6a14de501494214)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:38.911512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:38.934019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:35:38.963759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '598ad12efb0a4e5ab6a14de501494214', 'request_id': 'req-67493a16-62c9-4466-a0e9-71102fe2d0b8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.969232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '598ad12efb0a4e5ab6a14de501494214', 'request_id': 'req-67493a16-62c9-4466-a0e9-71102fe2d0b8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.978255 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '598ad12efb0a4e5ab6a14de501494214', 'request_id': 'req-67493a16-62c9-4466-a0e9-71102fe2d0b8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.990867 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] The token cache is being invalidate because project 598ad12efb0a4e5ab6a14de501494214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:35:38.991120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.991894 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:35:38.993582 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1365/2758] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:35:38 2022] DELETE /identity/v3/projects/598ad12efb0a4e5ab6a14de501494214 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:35:48.648984 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:35:48.986730 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:35:49.033990 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:35:49.035544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:49.101628 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae6e31-016b-4a8d-8ba5-e3c994dfd85b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:35:49.162329 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:35:49.162952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:35:49.163266 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:35:49.163535 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:35:49.165729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:35:49.176362 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:35:49.177802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:35:49.186194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1394/2759] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:35:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:35:59.182656 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=46 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:36:38.626574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:36:38.628086 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:36:38.640869 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67493a16-62c9-4466-a0e9-71102fe2d0b8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:36:38.644892 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:36:38.645640 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:36:38.645926 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:36:38.646204 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:36:38.648102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:36:38.658089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:36:38.659573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:36:38.668102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1366/2760] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:36:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:36:38.783561 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:36:38.792617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3043d7-2afe-4488-afe4-1febbd610c74 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:36:38.796495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecae9cf-8a05-4d07-894b-3082364a4da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:36:38.797132 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae9cf-8a05-4d07-894b-3082364a4da7 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:36:38.797410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae9cf-8a05-4d07-894b-3082364a4da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:36:38.797680 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae9cf-8a05-4d07-894b-3082364a4da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:36:38.799614 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3ecae9cf-8a05-4d07-894b-3082364a4da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:36:38.875898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecae9cf-8a05-4d07-894b-3082364a4da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:36:38.877370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecae9cf-8a05-4d07-894b-3082364a4da7 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:36:38.888857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1395/2761] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:36:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:36:48.664600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:36:48.884336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=34 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:36:58.071724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:36:58.073188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:36:58.090359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e176a43-105c-4ddc-bc40-e9d5f694219a service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:36:58.096177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:36:58.097019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:36:58.097304 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:36:58.097608 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:36:58.099643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:36:58.114303 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:36:58.117045 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:36:58.135401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1367/2762] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:36:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:37:00.917453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0f89-46d7-4427-b360-18bd5e61c0a9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:37:00.918039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0f89-46d7-4427-b360-18bd5e61c0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:37:00.918039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0f89-46d7-4427-b360-18bd5e61c0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:37:00.936101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b5fd0f89-46d7-4427-b360-18bd5e61c0a9 None None] MFA Rules not processed for user `4ab1ccac328d47c99119b8205cabfedd`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:37:00.937315 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5fd0f89-46d7-4427-b360-18bd5e61c0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:37:00.998851 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1396/2763] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:37:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:37:08.125839 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:37:10.988975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:37:12.270655 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:37:12.286475 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5318f8-0dfc-4143-bc43-387717d578b2 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:37:12.296308 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96456bfed82d469b888bd495ec46e796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:37:12.297171 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:37:12.297466 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:37:12.297738 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:37:12.299932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:37:12.312334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:37:12.314124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:37:12.323702 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1368/2764] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:37:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:37:22.319517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:29.187922 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd0f89-46d7-4427-b360-18bd5e61c0a9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:29.189590 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5fd0f89-46d7-4427-b360-18bd5e61c0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:29.201869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd0f89-46d7-4427-b360-18bd5e61c0a9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:29.206073 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:29.206776 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:29.207054 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:29.207564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:29.209574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:29.281508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:29.282958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:29.291729 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1397/2765] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:38:33.042266 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:33.044088 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:33.115662 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd749318-275a-4b3a-a2d0-49915030cb2f service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:33.122185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:33.122841 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:33.123154 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:33.123463 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:33.125669 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:33.138220 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:33.139871 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:33.150062 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1369/2766] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:38:34.052547 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:34.065510 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f8cc9-aeb7-4212-a24f-02779fe5e580 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:34.071948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:34.072833 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:34.073121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:34.073445 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:34.076123 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:34.144823 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:34.146306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:34.156531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1398/2767] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:38:37.361373 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:37.362826 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:37.376935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc1b99-9f39-4462-8c93-aedacdc7d691 service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:37.382585 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:37.383329 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:37.383682 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:37.383970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:37.385809 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:37.398209 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:37.399855 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:37.414717 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1370/2768] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:38:40.521115 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:40.523765 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:40.541719 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5d304-8f60-48f1-8ea8-040ce1efbe84 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:40.544933 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:40.545896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:40.546240 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:40.546627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] PATH_INFO: `/v3/users/1eb79dd547c04d2eac6dc1e35311bd9c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:40.547904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb79dd547c04d2eac6dc1e35311bd9c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:40.550391 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:40.603838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb79dd547c04d2eac6dc1e35311bd9c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:40.604448 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb79dd547c04d2eac6dc1e35311bd9c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:40.611868 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb79dd547c04d2eac6dc1e35311bd9c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:40.623894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb79dd547c04d2eac6dc1e35311bd9c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:40.633352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb79dd547c04d2eac6dc1e35311bd9c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:40.639847 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1399/2769] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:38:40 2022] DELETE /identity/v3/users/1eb79dd547c04d2eac6dc1e35311bd9c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:41.084339 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:41.097547 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f6c8-3e02-4ff5-a6d7-2ee700df4714 service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:41.157928 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:41.158801 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:41.159077 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:41.159376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] PATH_INFO: `/v3/projects/86a447448dde4ecd80717993f58f33a7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:41.160675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86a447448dde4ecd80717993f58f33a7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:41.162593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:41.220302 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86a447448dde4ecd80717993f58f33a7', 'request_id': 'req-c934a48b-0f01-41a0-b926-4d408d05d110'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.225952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86a447448dde4ecd80717993f58f33a7', 'request_id': 'req-c934a48b-0f01-41a0-b926-4d408d05d110'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.234401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86a447448dde4ecd80717993f58f33a7', 'request_id': 'req-c934a48b-0f01-41a0-b926-4d408d05d110'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.246460 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] The token cache is being invalidate because project 86a447448dde4ecd80717993f58f33a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:38:41.246728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.247541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.249226 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1371/2770] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:38:41 2022] DELETE /identity/v3/projects/86a447448dde4ecd80717993f58f33a7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:41.253603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:41.255106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:41.265296 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:41.269358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d111810-ac55-4ff4-91be-933b6808166d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:41.330177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:41.331130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:41.331557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:41.331828 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] PATH_INFO: `/v3/users/6d07a2cec61641408a74413b57711149` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:41.333021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d07a2cec61641408a74413b57711149)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:41.334962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:41.387385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d07a2cec61641408a74413b57711149'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.388486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d07a2cec61641408a74413b57711149'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.395498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d07a2cec61641408a74413b57711149'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.409358 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d07a2cec61641408a74413b57711149'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.418373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d07a2cec61641408a74413b57711149'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.424876 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1400/2771] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:38:41 2022] DELETE /identity/v3/users/6d07a2cec61641408a74413b57711149 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:41.614244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:41.615900 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:41.630620 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934a48b-0f01-41a0-b926-4d408d05d110 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:41.636357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:41.703595 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:41.704484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:41.704768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:41.705048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] PATH_INFO: `/v3/projects/102891aa9e4c42e589476bd2b684948b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:41.706231 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=102891aa9e4c42e589476bd2b684948b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:41.708254 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:41.755796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '102891aa9e4c42e589476bd2b684948b', 'request_id': 'req-586ac937-7edf-4a5a-b922-a2f3346785b8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.761301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '102891aa9e4c42e589476bd2b684948b', 'request_id': 'req-586ac937-7edf-4a5a-b922-a2f3346785b8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.769800 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '102891aa9e4c42e589476bd2b684948b', 'request_id': 'req-586ac937-7edf-4a5a-b922-a2f3346785b8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.780922 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] The token cache is being invalidate because project 102891aa9e4c42e589476bd2b684948b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:38:41.781210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.781838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:41.783456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1372/2772] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:38:41 2022] DELETE /identity/v3/projects/102891aa9e4c42e589476bd2b684948b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:48.107965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:48.109549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:48.173222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25a767-5bcf-423d-a65b-2983f3120605 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:48.211091 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:48.225958 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:48.226617 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:48.226899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:48.227171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:48.229161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:48.293138 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:48.294735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:48.305334 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1401/2773] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:38:51.433119 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:51.434664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:51.449556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ac937-7edf-4a5a-b922-a2f3346785b8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:51.454143 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:51.455078 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:51.455388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:51.455661 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] PATH_INFO: `/v3/users/4ab1ccac328d47c99119b8205cabfedd` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:51.456843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab1ccac328d47c99119b8205cabfedd)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:51.458768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:51.518048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab1ccac328d47c99119b8205cabfedd'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.528425 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab1ccac328d47c99119b8205cabfedd'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.536070 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab1ccac328d47c99119b8205cabfedd'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.536797 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab1ccac328d47c99119b8205cabfedd'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.549263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-12d61b45-9618-4924-8378-61be68191725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab1ccac328d47c99119b8205cabfedd'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.555426 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1373/2774] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:38:51 2022] DELETE /identity/v3/users/4ab1ccac328d47c99119b8205cabfedd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:51.741840 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:51.752934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ce8de-e89a-4104-b4b2-0b33bb9969cb service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:51.812496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:51.813385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:51.813657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:51.813956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] PATH_INFO: `/v3/projects/e27862b1b0f2434d86d6af9998716432` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:51.815144 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27862b1b0f2434d86d6af9998716432)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:51.817098 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:51.865925 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27862b1b0f2434d86d6af9998716432', 'request_id': 'req-43638071-74b5-456e-9953-25da9197a532'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.871657 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27862b1b0f2434d86d6af9998716432', 'request_id': 'req-43638071-74b5-456e-9953-25da9197a532'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.880742 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27862b1b0f2434d86d6af9998716432', 'request_id': 'req-43638071-74b5-456e-9953-25da9197a532'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.892467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] The token cache is being invalidate because project e27862b1b0f2434d86d6af9998716432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:38:51.892751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.893374 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43638071-74b5-456e-9953-25da9197a532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:51.894938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1402/2775] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:38:51 2022] DELETE /identity/v3/projects/e27862b1b0f2434d86d6af9998716432 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:51.900058 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1376b8-0f8b-4143-b30f-faad3b3c4561 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:51.900376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1376b8-0f8b-4143-b30f-faad3b3c4561 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:51.900649 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1376b8-0f8b-4143-b30f-faad3b3c4561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:51.912043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:51.959947 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7f1376b8-0f8b-4143-b30f-faad3b3c4561 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:51.961190 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7f1376b8-0f8b-4143-b30f-faad3b3c4561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:52.023865 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1374/2776] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:52.032105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa743884-a7ae-405a-9326-0e33fc4f3bdd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.032386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa743884-a7ae-405a-9326-0e33fc4f3bdd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.032654 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa743884-a7ae-405a-9326-0e33fc4f3bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.052056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fa743884-a7ae-405a-9326-0e33fc4f3bdd None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:52.053975 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fa743884-a7ae-405a-9326-0e33fc4f3bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:52.067038 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1403/2777] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:52.071524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1376b8-0f8b-4143-b30f-faad3b3c4561 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.083038 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1376b8-0f8b-4143-b30f-faad3b3c4561 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.089537 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.090205 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.090490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.090763 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.091534 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:38:52.092079 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.094052 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.107279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1375/2778] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:38:52.112168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa743884-a7ae-405a-9326-0e33fc4f3bdd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.121905 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa743884-a7ae-405a-9326-0e33fc4f3bdd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.130275 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6d17e-09e6-4328-846e-6d3d72005931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.131290 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6d17e-09e6-4328-846e-6d3d72005931 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.131678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6d17e-09e6-4328-846e-6d3d72005931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.132072 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6d17e-09e6-4328-846e-6d3d72005931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.133067 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6d17e-09e6-4328-846e-6d3d72005931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.136122 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6d17e-09e6-4328-846e-6d3d72005931 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.151450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:52.184221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1404/2779] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:38:52 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:52.188976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.199135 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38bda2-1647-4de8-a3e0-26a94b8a6b0a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.255609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c825a-ccf0-4631-9262-60d4bfc26599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.256271 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c825a-ccf0-4631-9262-60d4bfc26599 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.256551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c825a-ccf0-4631-9262-60d4bfc26599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.256825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c825a-ccf0-4631-9262-60d4bfc26599 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.257487 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c825a-ccf0-4631-9262-60d4bfc26599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.259457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c825a-ccf0-4631-9262-60d4bfc26599 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.281451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1376/2780] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:38:52 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:52.285629 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6d17e-09e6-4328-846e-6d3d72005931 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.296945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6d17e-09e6-4328-846e-6d3d72005931 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.305458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0f7da-e8ec-41ae-be22-07b7ed8d622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.306385 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0f7da-e8ec-41ae-be22-07b7ed8d622a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.306743 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0f7da-e8ec-41ae-be22-07b7ed8d622a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.307074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0f7da-e8ec-41ae-be22-07b7ed8d622a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.308248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0f7da-e8ec-41ae-be22-07b7ed8d622a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.310704 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0f7da-e8ec-41ae-be22-07b7ed8d622a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.321468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1405/2781] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:52.325428 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c825a-ccf0-4631-9262-60d4bfc26599 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.336522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c825a-ccf0-4631-9262-60d4bfc26599 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.343140 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524b29-3027-43a6-bf0b-1f870b3e9567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.343846 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524b29-3027-43a6-bf0b-1f870b3e9567 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.344116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524b29-3027-43a6-bf0b-1f870b3e9567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.344424 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524b29-3027-43a6-bf0b-1f870b3e9567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.345130 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524b29-3027-43a6-bf0b-1f870b3e9567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.346737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524b29-3027-43a6-bf0b-1f870b3e9567 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.357267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1377/2782] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:52.361450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0f7da-e8ec-41ae-be22-07b7ed8d622a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.372483 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0f7da-e8ec-41ae-be22-07b7ed8d622a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.378678 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d752b5-10d8-484a-a128-db431a8e19bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.379622 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d752b5-10d8-484a-a128-db431a8e19bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.379896 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d752b5-10d8-484a-a128-db431a8e19bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.380169 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d752b5-10d8-484a-a128-db431a8e19bb admin admin] PATH_INFO: `/v3/projects/4bd08ba692194591a008ffef408ee194/users/db9a1054fe9b485da3b08c38bf7a34b6/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.396511 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d752b5-10d8-484a-a128-db431a8e19bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd08ba692194591a008ffef408ee194, user_id=db9a1054fe9b485da3b08c38bf7a34b6, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.399193 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d752b5-10d8-484a-a128-db431a8e19bb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.417587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1406/2783] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:38:52 2022] PUT /identity/v3/projects/4bd08ba692194591a008ffef408ee194/users/db9a1054fe9b485da3b08c38bf7a34b6/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:52.428915 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03889f-bae9-4ea5-986c-41d6859e9f7c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.429312 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03889f-bae9-4ea5-986c-41d6859e9f7c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.429670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03889f-bae9-4ea5-986c-41d6859e9f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.460816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cc03889f-bae9-4ea5-986c-41d6859e9f7c None None] MFA Rules not processed for user `db9a1054fe9b485da3b08c38bf7a34b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:52.461978 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cc03889f-bae9-4ea5-986c-41d6859e9f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:52.499824 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:52.539491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1378/2784] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:52.544113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d752b5-10d8-484a-a128-db431a8e19bb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.606191 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d752b5-10d8-484a-a128-db431a8e19bb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.612064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9dee-45f5-4444-ac2f-02fa52f74e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.612713 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9dee-45f5-4444-ac2f-02fa52f74e5d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.612994 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9dee-45f5-4444-ac2f-02fa52f74e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.613264 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9dee-45f5-4444-ac2f-02fa52f74e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.613942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf9dee-45f5-4444-ac2f-02fa52f74e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.615898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf9dee-45f5-4444-ac2f-02fa52f74e5d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.648542 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1407/2785] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:38:52 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:52.652632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03889f-bae9-4ea5-986c-41d6859e9f7c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.664006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03889f-bae9-4ea5-986c-41d6859e9f7c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.722704 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdcf59-88db-4804-9ac1-d966c856ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.723327 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdcf59-88db-4804-9ac1-d966c856ac13 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.723604 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdcf59-88db-4804-9ac1-d966c856ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.723881 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdcf59-88db-4804-9ac1-d966c856ac13 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.724589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cdcf59-88db-4804-9ac1-d966c856ac13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.726501 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cdcf59-88db-4804-9ac1-d966c856ac13 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.749924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1379/2786] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:38:52 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:52.754121 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9dee-45f5-4444-ac2f-02fa52f74e5d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.765184 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9dee-45f5-4444-ac2f-02fa52f74e5d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.770695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006d35d-c78f-4c8e-8f9e-f4db3081537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.771386 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006d35d-c78f-4c8e-8f9e-f4db3081537b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.771688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006d35d-c78f-4c8e-8f9e-f4db3081537b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.771959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006d35d-c78f-4c8e-8f9e-f4db3081537b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.772688 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006d35d-c78f-4c8e-8f9e-f4db3081537b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.774373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006d35d-c78f-4c8e-8f9e-f4db3081537b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.785252 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1408/2787] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:52.789615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdcf59-88db-4804-9ac1-d966c856ac13 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.799728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdcf59-88db-4804-9ac1-d966c856ac13 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.804818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1be2bb-27e9-410b-86a9-a9bb902be8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.805712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1be2bb-27e9-410b-86a9-a9bb902be8ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.805992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1be2bb-27e9-410b-86a9-a9bb902be8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.806261 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1be2bb-27e9-410b-86a9-a9bb902be8ae admin admin] PATH_INFO: `/v3/projects/c3ed3f7a8025421ab270ad1d578bf97b/users/5d47a4b1baa84bdebb9bfc336b611a9d/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.821972 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1be2bb-27e9-410b-86a9-a9bb902be8ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ed3f7a8025421ab270ad1d578bf97b, user_id=5d47a4b1baa84bdebb9bfc336b611a9d, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.824748 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1be2bb-27e9-410b-86a9-a9bb902be8ae admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.842314 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1380/2788] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:38:52 2022] PUT /identity/v3/projects/c3ed3f7a8025421ab270ad1d578bf97b/users/5d47a4b1baa84bdebb9bfc336b611a9d/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:52.846468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006d35d-c78f-4c8e-8f9e-f4db3081537b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.856698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006d35d-c78f-4c8e-8f9e-f4db3081537b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.914983 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa11b3-3be0-453d-a6d4-b0e0a419929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.915664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa11b3-3be0-453d-a6d4-b0e0a419929b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.915968 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa11b3-3be0-453d-a6d4-b0e0a419929b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.916247 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa11b3-3be0-453d-a6d4-b0e0a419929b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.917005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa11b3-3be0-453d-a6d4-b0e0a419929b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.918638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa11b3-3be0-453d-a6d4-b0e0a419929b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.929075 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1409/2789] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:52.933359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1be2bb-27e9-410b-86a9-a9bb902be8ae admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:52.943850 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1be2bb-27e9-410b-86a9-a9bb902be8ae admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:52.949064 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:52.949955 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.950232 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.950532 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] PATH_INFO: `/v3/projects/c3ed3f7a8025421ab270ad1d578bf97b/users/5d47a4b1baa84bdebb9bfc336b611a9d/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:52.952967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ed3f7a8025421ab270ad1d578bf97b, user_id=5d47a4b1baa84bdebb9bfc336b611a9d, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:52.955653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:52.966622 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1381/2790] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:38:52 2022] PUT /identity/v3/projects/c3ed3f7a8025421ab270ad1d578bf97b/users/5d47a4b1baa84bdebb9bfc336b611a9d/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:52.978280 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8dc27-5f18-48fe-83cb-46f939badaef None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:52.978481 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8dc27-5f18-48fe-83cb-46f939badaef None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:52.978771 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8dc27-5f18-48fe-83cb-46f939badaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:53.006651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-31a8dc27-5f18-48fe-83cb-46f939badaef None None] MFA Rules not processed for user `5d47a4b1baa84bdebb9bfc336b611a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:53.008025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-31a8dc27-5f18-48fe-83cb-46f939badaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:53.013262 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:53.083174 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1410/2791] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:53.095940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:53.098141 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:53.179834 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4945f-640e-4d78-8c7c-20245f39d82a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:53.184454 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f8bcc-eb1a-4a2d-85d5-c574c71921fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:53.185110 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8bcc-eb1a-4a2d-85d5-c574c71921fa service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:53.185393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8bcc-eb1a-4a2d-85d5-c574c71921fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:53.185827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8bcc-eb1a-4a2d-85d5-c574c71921fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:53.196545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f8bcc-eb1a-4a2d-85d5-c574c71921fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:53.198009 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f8bcc-eb1a-4a2d-85d5-c574c71921fa service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:53.208529 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1382/2792] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:38:53.561022 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8dc27-5f18-48fe-83cb-46f939badaef None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:53.571035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8dc27-5f18-48fe-83cb-46f939badaef None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:53.576693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d47a4b1baa84bdebb9bfc336b611a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ed3f7a8025421ab270ad1d578bf97b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:53.577383 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:53.577670 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:53.577963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:53.578695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:38:53.579004 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:53.580660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:53.592513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1411/2793] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:38:53.597146 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f8bcc-eb1a-4a2d-85d5-c574c71921fa service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:53.606573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f8bcc-eb1a-4a2d-85d5-c574c71921fa service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:53.611878 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1732fb-89a6-465b-8eea-c011e78533bb tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d47a4b1baa84bdebb9bfc336b611a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ed3f7a8025421ab270ad1d578bf97b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:53.612538 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1732fb-89a6-465b-8eea-c011e78533bb tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:53.612821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1732fb-89a6-465b-8eea-c011e78533bb tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:53.613097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1732fb-89a6-465b-8eea-c011e78533bb tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:53.613795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1732fb-89a6-465b-8eea-c011e78533bb tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:53.615745 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1732fb-89a6-465b-8eea-c011e78533bb tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:53.620107 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:53.644958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1383/2794] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:38:53 2022] POST /identity/v3/projects => generated 354 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:53.649468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:53.660348 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2235a9-54b2-40bc-a5ae-50335877f74b tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:53.724582 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d47a4b1baa84bdebb9bfc336b611a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ed3f7a8025421ab270ad1d578bf97b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:53.725245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:53.725532 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:53.725807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:53.726536 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:38:53.726848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:53.728513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:53.740589 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1412/2795] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:38:53.776516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1732fb-89a6-465b-8eea-c011e78533bb tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:53.787397 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1732fb-89a6-465b-8eea-c011e78533bb tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:53.792765 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d47a4b1baa84bdebb9bfc336b611a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ed3f7a8025421ab270ad1d578bf97b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:53.793636 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:53.794056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:53.794336 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] PATH_INFO: `/v3/projects/fe39c7d712714984b898408ecc1db78c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:53.795569 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fe39c7d712714984b898408ecc1db78c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:53.797504 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:53.845385 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe39c7d712714984b898408ecc1db78c', 'request_id': 'req-9afe51dd-012e-4008-bf60-26899cdc3c9d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:53.850719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe39c7d712714984b898408ecc1db78c', 'request_id': 'req-9afe51dd-012e-4008-bf60-26899cdc3c9d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:53.860337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe39c7d712714984b898408ecc1db78c', 'request_id': 'req-9afe51dd-012e-4008-bf60-26899cdc3c9d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:53.872729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] The token cache is being invalidate because project fe39c7d712714984b898408ecc1db78c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:38:53.873018 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:53.873695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9afe51dd-012e-4008-bf60-26899cdc3c9d tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:53.875337 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1384/2796] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:38:53 2022] DELETE /identity/v3/projects/fe39c7d712714984b898408ecc1db78c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:54.089009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:54.091021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:54.157881 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd01ec-c64e-4e1e-af6a-2da73fbca846 tempest-VolumeQuotaClassesTest-1085857222 tempest-VolumeQuotaClassesTest-1085857222-project-admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:54.187450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:54.215802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:54.216689 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:54.216972 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:54.217248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] PATH_INFO: `/v3/users/db9a1054fe9b485da3b08c38bf7a34b6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:54.218496 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9a1054fe9b485da3b08c38bf7a34b6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:54.220468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:54.270287 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9a1054fe9b485da3b08c38bf7a34b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.270904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9a1054fe9b485da3b08c38bf7a34b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.277554 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9a1054fe9b485da3b08c38bf7a34b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.289732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9a1054fe9b485da3b08c38bf7a34b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.298436 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9a1054fe9b485da3b08c38bf7a34b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.304434 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1413/2797] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:38:54 2022] DELETE /identity/v3/users/db9a1054fe9b485da3b08c38bf7a34b6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:54.309120 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f23ede-1041-4039-8c6a-1b1405666e82 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:54.309413 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f23ede-1041-4039-8c6a-1b1405666e82 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:54.309697 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f23ede-1041-4039-8c6a-1b1405666e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:54.327643 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-98f23ede-1041-4039-8c6a-1b1405666e82 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:54.328981 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-98f23ede-1041-4039-8c6a-1b1405666e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:54.392332 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1385/2798] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:54.402221 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:54.403836 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:54.464578 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31939a3c-b521-4bb3-b307-e4b917d5b036 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:54.469453 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e533e6a-e89b-4831-b515-c0e37955c1a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:54.470088 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533e6a-e89b-4831-b515-c0e37955c1a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:54.470373 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533e6a-e89b-4831-b515-c0e37955c1a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:54.470644 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533e6a-e89b-4831-b515-c0e37955c1a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:54.481820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e533e6a-e89b-4831-b515-c0e37955c1a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:54.483338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e533e6a-e89b-4831-b515-c0e37955c1a5 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:54.493959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1414/2799] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:38:54.770805 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f23ede-1041-4039-8c6a-1b1405666e82 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:54.772605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-98f23ede-1041-4039-8c6a-1b1405666e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:54.787234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f23ede-1041-4039-8c6a-1b1405666e82 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:54.792453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:54.793525 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:54.793780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:54.794054 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] PATH_INFO: `/v3/projects/4bd08ba692194591a008ffef408ee194` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:54.795279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd08ba692194591a008ffef408ee194)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:54.797248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:54.846667 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd08ba692194591a008ffef408ee194', 'request_id': 'req-a91bd55b-9c84-4bc0-bd6e-da706fc61990'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.852049 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd08ba692194591a008ffef408ee194', 'request_id': 'req-a91bd55b-9c84-4bc0-bd6e-da706fc61990'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.860759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd08ba692194591a008ffef408ee194', 'request_id': 'req-a91bd55b-9c84-4bc0-bd6e-da706fc61990'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.861885 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:54.874108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] The token cache is being invalidate because project 4bd08ba692194591a008ffef408ee194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:38:54.874747 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.876025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:54.877840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1386/2800] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:38:54 2022] DELETE /identity/v3/projects/4bd08ba692194591a008ffef408ee194 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:54.881903 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e533e6a-e89b-4831-b515-c0e37955c1a5 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:54.891989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e533e6a-e89b-4831-b515-c0e37955c1a5 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:54.939838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:54.950458 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:54.951398 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:54.951699 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:54.951978 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] PATH_INFO: `/v3/users/5d47a4b1baa84bdebb9bfc336b611a9d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:54.953869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d47a4b1baa84bdebb9bfc336b611a9d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:54.956128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:55.007947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d47a4b1baa84bdebb9bfc336b611a9d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.008592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d47a4b1baa84bdebb9bfc336b611a9d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.016859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d47a4b1baa84bdebb9bfc336b611a9d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.032750 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d47a4b1baa84bdebb9bfc336b611a9d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.043214 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11209f83-22ff-4b97-ba34-9b38c5772c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d47a4b1baa84bdebb9bfc336b611a9d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.049859 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1415/2801] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:38:54 2022] DELETE /identity/v3/users/5d47a4b1baa84bdebb9bfc336b611a9d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:55.344248 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:55.346357 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:55.360884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bd55b-9c84-4bc0-bd6e-da706fc61990 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:55.419361 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:55.420242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.420546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.420821 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] PATH_INFO: `/v3/projects/c3ed3f7a8025421ab270ad1d578bf97b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.422252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3ed3f7a8025421ab270ad1d578bf97b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:55.424244 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:55.472675 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ed3f7a8025421ab270ad1d578bf97b', 'request_id': 'req-ce20111d-66e9-45c0-b80f-9aba67ad754d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.478365 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ed3f7a8025421ab270ad1d578bf97b', 'request_id': 'req-ce20111d-66e9-45c0-b80f-9aba67ad754d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.487806 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ed3f7a8025421ab270ad1d578bf97b', 'request_id': 'req-ce20111d-66e9-45c0-b80f-9aba67ad754d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.500436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] The token cache is being invalidate because project c3ed3f7a8025421ab270ad1d578bf97b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:38:55.500726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.501444 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ce20111d-66e9-45c0-b80f-9aba67ad754d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:38:55.503093 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1387/2802] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:38:55 2022] DELETE /identity/v3/projects/c3ed3f7a8025421ab270ad1d578bf97b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:55.508080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549a8c1-f637-4f03-ad51-ef89cd343e0f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.508586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549a8c1-f637-4f03-ad51-ef89cd343e0f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.508708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549a8c1-f637-4f03-ad51-ef89cd343e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.526977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2549a8c1-f637-4f03-ad51-ef89cd343e0f None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:55.528180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2549a8c1-f637-4f03-ad51-ef89cd343e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:55.596341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1416/2803] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:55.604310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe1e2f-806f-4176-8eb1-4347d1d6e952 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.604597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe1e2f-806f-4176-8eb1-4347d1d6e952 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.604891 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe1e2f-806f-4176-8eb1-4347d1d6e952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.623886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1bfe1e2f-806f-4176-8eb1-4347d1d6e952 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:55.625118 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1bfe1e2f-806f-4176-8eb1-4347d1d6e952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:55.634032 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1388/2804] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:55.638378 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549a8c1-f637-4f03-ad51-ef89cd343e0f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:55.648857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549a8c1-f637-4f03-ad51-ef89cd343e0f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:55.653608 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:55.654352 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.654705 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.655048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.656142 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:38:55.656274 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:55.658239 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:55.670898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1417/2805] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:38:55.675090 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe1e2f-806f-4176-8eb1-4347d1d6e952 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:55.684736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe1e2f-806f-4176-8eb1-4347d1d6e952 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:55.690096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f27a4-8d83-4417-b2ff-481fdcfaedaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:55.690858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f27a4-8d83-4417-b2ff-481fdcfaedaf admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.691234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f27a4-8d83-4417-b2ff-481fdcfaedaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.691600 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f27a4-8d83-4417-b2ff-481fdcfaedaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.692411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f27a4-8d83-4417-b2ff-481fdcfaedaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:55.694415 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f27a4-8d83-4417-b2ff-481fdcfaedaf admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:55.721961 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1389/2806] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:38:55 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:38:55.726357 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:55.737267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f008fe-83fc-4564-b8e1-2cd335d91100 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:55.794344 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f99d14-f9fb-4f72-93e6-81b04b1663b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:55.795025 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f99d14-f9fb-4f72-93e6-81b04b1663b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.795338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f99d14-f9fb-4f72-93e6-81b04b1663b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.795618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f99d14-f9fb-4f72-93e6-81b04b1663b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.796322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f99d14-f9fb-4f72-93e6-81b04b1663b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:55.798366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f99d14-f9fb-4f72-93e6-81b04b1663b5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:55.805323 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:55.821394 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1418/2807] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:38:55 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:55.825457 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f27a4-8d83-4417-b2ff-481fdcfaedaf admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:55.836436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f27a4-8d83-4417-b2ff-481fdcfaedaf admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:55.840975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8df881-3e35-4bb7-bf8c-44a6884675b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:55.841645 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8df881-3e35-4bb7-bf8c-44a6884675b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.841931 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8df881-3e35-4bb7-bf8c-44a6884675b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.842210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8df881-3e35-4bb7-bf8c-44a6884675b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.843076 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8df881-3e35-4bb7-bf8c-44a6884675b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:55.844762 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8df881-3e35-4bb7-bf8c-44a6884675b3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:55.855526 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1390/2808] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:55.859923 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f99d14-f9fb-4f72-93e6-81b04b1663b5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:55.870778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f99d14-f9fb-4f72-93e6-81b04b1663b5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:55.876467 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8448c88-efb2-46f9-8f14-5a636213dc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:55.877062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8448c88-efb2-46f9-8f14-5a636213dc06 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.877341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8448c88-efb2-46f9-8f14-5a636213dc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.877727 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8448c88-efb2-46f9-8f14-5a636213dc06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.878468 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8448c88-efb2-46f9-8f14-5a636213dc06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:55.880130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8448c88-efb2-46f9-8f14-5a636213dc06 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:55.890564 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1419/2809] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:38:55.894842 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8df881-3e35-4bb7-bf8c-44a6884675b3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:55.904844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8df881-3e35-4bb7-bf8c-44a6884675b3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:55.909638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaca005-92e0-4a70-ad24-0b8f2152a9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:55.910558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaca005-92e0-4a70-ad24-0b8f2152a9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.910838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaca005-92e0-4a70-ad24-0b8f2152a9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.911108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaca005-92e0-4a70-ad24-0b8f2152a9f7 admin admin] PATH_INFO: `/v3/projects/76523ff80071488a8f4c0e7a4596cff5/users/138942cb91954ab8acfe9e42f760fbc2/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.925388 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaca005-92e0-4a70-ad24-0b8f2152a9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76523ff80071488a8f4c0e7a4596cff5, user_id=138942cb91954ab8acfe9e42f760fbc2, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:55.928056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaca005-92e0-4a70-ad24-0b8f2152a9f7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:55.943001 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1391/2810] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:38:55 2022] PUT /identity/v3/projects/76523ff80071488a8f4c0e7a4596cff5/users/138942cb91954ab8acfe9e42f760fbc2/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:38:55.953158 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde9ef-90c5-4fce-9475-c0cb8a68c0d3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:55.953465 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde9ef-90c5-4fce-9475-c0cb8a68c0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:55.953904 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde9ef-90c5-4fce-9475-c0cb8a68c0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:55.981245 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-43bde9ef-90c5-4fce-9475-c0cb8a68c0d3 None None] MFA Rules not processed for user `138942cb91954ab8acfe9e42f760fbc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:55.982433 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-43bde9ef-90c5-4fce-9475-c0cb8a68c0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:56.050977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1420/2811] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:56.055849 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaca005-92e0-4a70-ad24-0b8f2152a9f7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.083516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:56.127166 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaca005-92e0-4a70-ad24-0b8f2152a9f7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.132575 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3489400-aa73-49d4-8dd5-a6a6f401418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.133242 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489400-aa73-49d4-8dd5-a6a6f401418e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.133518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489400-aa73-49d4-8dd5-a6a6f401418e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.133787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489400-aa73-49d4-8dd5-a6a6f401418e admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.134480 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3489400-aa73-49d4-8dd5-a6a6f401418e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.136436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3489400-aa73-49d4-8dd5-a6a6f401418e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.168408 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1392/2812] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:38:56 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:38:56.172708 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bde9ef-90c5-4fce-9475-c0cb8a68c0d3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.184217 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bde9ef-90c5-4fce-9475-c0cb8a68c0d3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.240055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c188313-2a77-43d4-913a-42eb449dc3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.240732 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c188313-2a77-43d4-913a-42eb449dc3bc admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.241009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c188313-2a77-43d4-913a-42eb449dc3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.241403 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c188313-2a77-43d4-913a-42eb449dc3bc admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.242095 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c188313-2a77-43d4-913a-42eb449dc3bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.244060 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c188313-2a77-43d4-913a-42eb449dc3bc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.266157 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1421/2813] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:38:56 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:56.270383 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3489400-aa73-49d4-8dd5-a6a6f401418e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.281162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3489400-aa73-49d4-8dd5-a6a6f401418e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.286793 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4799a3-71be-4cf5-a6d8-dd7940000e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.287515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4799a3-71be-4cf5-a6d8-dd7940000e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.287794 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4799a3-71be-4cf5-a6d8-dd7940000e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.288066 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4799a3-71be-4cf5-a6d8-dd7940000e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.288799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4799a3-71be-4cf5-a6d8-dd7940000e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.290416 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4799a3-71be-4cf5-a6d8-dd7940000e47 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.301770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1393/2814] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:56.305789 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c188313-2a77-43d4-913a-42eb449dc3bc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.316171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c188313-2a77-43d4-913a-42eb449dc3bc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.321499 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e3503-412c-46de-8719-774fe06f4f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.322154 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e3503-412c-46de-8719-774fe06f4f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.322432 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e3503-412c-46de-8719-774fe06f4f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.322711 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e3503-412c-46de-8719-774fe06f4f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.323479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e3503-412c-46de-8719-774fe06f4f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.325076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e3503-412c-46de-8719-774fe06f4f0b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.335731 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1422/2815] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:56.339906 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4799a3-71be-4cf5-a6d8-dd7940000e47 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.350833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4799a3-71be-4cf5-a6d8-dd7940000e47 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.356728 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d504-cf98-4e74-ad0e-6463c87c18df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.357654 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d504-cf98-4e74-ad0e-6463c87c18df admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.357935 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d504-cf98-4e74-ad0e-6463c87c18df admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.358203 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d504-cf98-4e74-ad0e-6463c87c18df admin admin] PATH_INFO: `/v3/projects/e894d8037e564371a6876173662960de/users/ac86ccb99b3747329d94fa39f3c5c9a9/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.374322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77d504-cf98-4e74-ad0e-6463c87c18df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e894d8037e564371a6876173662960de, user_id=ac86ccb99b3747329d94fa39f3c5c9a9, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.377056 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77d504-cf98-4e74-ad0e-6463c87c18df admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.392688 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1394/2816] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:38:56 2022] PUT /identity/v3/projects/e894d8037e564371a6876173662960de/users/ac86ccb99b3747329d94fa39f3c5c9a9/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:56.404271 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a7f64-9bd2-42a1-8ecc-07d32c1234d8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.404557 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a7f64-9bd2-42a1-8ecc-07d32c1234d8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.404852 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a7f64-9bd2-42a1-8ecc-07d32c1234d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.436064 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5f8a7f64-9bd2-42a1-8ecc-07d32c1234d8 None None] MFA Rules not processed for user `ac86ccb99b3747329d94fa39f3c5c9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:56.437291 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5f8a7f64-9bd2-42a1-8ecc-07d32c1234d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:56.478827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:56.513580 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1423/2817] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:56.518398 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d504-cf98-4e74-ad0e-6463c87c18df admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.580393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d504-cf98-4e74-ad0e-6463c87c18df admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.585642 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dac49-b648-4988-a950-a8c4af93b627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.586290 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dac49-b648-4988-a950-a8c4af93b627 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.586568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dac49-b648-4988-a950-a8c4af93b627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.586843 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dac49-b648-4988-a950-a8c4af93b627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.587589 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dac49-b648-4988-a950-a8c4af93b627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.589491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dac49-b648-4988-a950-a8c4af93b627 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.622073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1395/2818] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:38:56 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:56.627351 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a7f64-9bd2-42a1-8ecc-07d32c1234d8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.639063 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a7f64-9bd2-42a1-8ecc-07d32c1234d8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.700606 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb536bfc-b8f8-48f4-8fde-50d39289daf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.701180 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536bfc-b8f8-48f4-8fde-50d39289daf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.701459 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536bfc-b8f8-48f4-8fde-50d39289daf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.701728 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536bfc-b8f8-48f4-8fde-50d39289daf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.702407 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb536bfc-b8f8-48f4-8fde-50d39289daf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.704396 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb536bfc-b8f8-48f4-8fde-50d39289daf5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.725134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1424/2819] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:38:56 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:38:56.729490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dac49-b648-4988-a950-a8c4af93b627 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.741160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dac49-b648-4988-a950-a8c4af93b627 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.746939 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca7c33-4dfa-4a6d-8fbd-133a5c483cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.747646 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca7c33-4dfa-4a6d-8fbd-133a5c483cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.747936 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca7c33-4dfa-4a6d-8fbd-133a5c483cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.748217 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca7c33-4dfa-4a6d-8fbd-133a5c483cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.748956 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca7c33-4dfa-4a6d-8fbd-133a5c483cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.751092 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca7c33-4dfa-4a6d-8fbd-133a5c483cbb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.761940 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1396/2820] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:56.766135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb536bfc-b8f8-48f4-8fde-50d39289daf5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.776319 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb536bfc-b8f8-48f4-8fde-50d39289daf5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.781106 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c660ab-5251-4847-a16f-44d3caecde0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.782058 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c660ab-5251-4847-a16f-44d3caecde0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.782332 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c660ab-5251-4847-a16f-44d3caecde0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.782595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c660ab-5251-4847-a16f-44d3caecde0b admin admin] PATH_INFO: `/v3/projects/433cbef76a124245b55d84f146399348/users/0949acc349bb4212a51131c0b448c16f/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.800137 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c660ab-5251-4847-a16f-44d3caecde0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=433cbef76a124245b55d84f146399348, user_id=0949acc349bb4212a51131c0b448c16f, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.802807 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c660ab-5251-4847-a16f-44d3caecde0b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.819952 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1425/2821] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:38:56 2022] PUT /identity/v3/projects/433cbef76a124245b55d84f146399348/users/0949acc349bb4212a51131c0b448c16f/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:38:56.824584 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca7c33-4dfa-4a6d-8fbd-133a5c483cbb admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.836278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca7c33-4dfa-4a6d-8fbd-133a5c483cbb admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.879924 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:38:56.899497 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb810e3-8538-4fb2-afe2-b4c3bbb8323a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.900176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb810e3-8538-4fb2-afe2-b4c3bbb8323a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.900456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb810e3-8538-4fb2-afe2-b4c3bbb8323a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.900729 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb810e3-8538-4fb2-afe2-b4c3bbb8323a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.901461 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb810e3-8538-4fb2-afe2-b4c3bbb8323a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.903061 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb810e3-8538-4fb2-afe2-b4c3bbb8323a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.913307 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1397/2822] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:38:56.917758 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c660ab-5251-4847-a16f-44d3caecde0b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:56.929008 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c660ab-5251-4847-a16f-44d3caecde0b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:56.934894 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:56.935848 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.936128 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.936399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] PATH_INFO: `/v3/projects/433cbef76a124245b55d84f146399348/users/0949acc349bb4212a51131c0b448c16f/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.938748 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=433cbef76a124245b55d84f146399348, user_id=0949acc349bb4212a51131c0b448c16f, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:56.941450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:56.952502 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1426/2823] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:38:56 2022] PUT /identity/v3/projects/433cbef76a124245b55d84f146399348/users/0949acc349bb4212a51131c0b448c16f/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:38:56.962769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda512d-7dc5-4c99-94ee-369d9d43d4a1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:56.963060 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda512d-7dc5-4c99-94ee-369d9d43d4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:56.963402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda512d-7dc5-4c99-94ee-369d9d43d4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:56.991081 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-feda512d-7dc5-4c99-94ee-369d9d43d4a1 None None] MFA Rules not processed for user `0949acc349bb4212a51131c0b448c16f`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:38:56.992347 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-feda512d-7dc5-4c99-94ee-369d9d43d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:57.065300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1398/2824] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:38:57.077016 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:57.079061 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:57.139715 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227889cd-88fd-4ba4-8ba0-be5c8866e1ba admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:57.144460 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3337a-a304-4c7b-b8e7-b82b332b259c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:57.145140 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3337a-a304-4c7b-b8e7-b82b332b259c service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:57.145408 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3337a-a304-4c7b-b8e7-b82b332b259c service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:57.145717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3337a-a304-4c7b-b8e7-b82b332b259c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:57.155171 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3337a-a304-4c7b-b8e7-b82b332b259c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:57.156694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3337a-a304-4c7b-b8e7-b82b332b259c service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:57.166055 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1427/2825] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:38:57.684473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda512d-7dc5-4c99-94ee-369d9d43d4a1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:38:57.686115 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-feda512d-7dc5-4c99-94ee-369d9d43d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:38:57.699953 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda512d-7dc5-4c99-94ee-369d9d43d4a1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:38:57.704819 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a58f-e865-4082-a915-4ca8318964a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:38:57.705473 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a58f-e865-4082-a915-4ca8318964a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:38:57.705759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a58f-e865-4082-a915-4ca8318964a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:38:57.706030 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a58f-e865-4082-a915-4ca8318964a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:38:57.772362 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5a58f-e865-4082-a915-4ca8318964a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:38:57.773715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5a58f-e865-4082-a915-4ca8318964a9 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:38:57.783825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1399/2826] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:38:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:01.882068 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3337a-a304-4c7b-b8e7-b82b332b259c service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:01.891046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3337a-a304-4c7b-b8e7-b82b332b259c service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:01.894867 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ef657-5e2a-424a-8e20-9c82e9492aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:01.895588 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef657-5e2a-424a-8e20-9c82e9492aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:01.896135 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef657-5e2a-424a-8e20-9c82e9492aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:01.896664 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef657-5e2a-424a-8e20-9c82e9492aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:01.927909 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:01.963321 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ef657-5e2a-424a-8e20-9c82e9492aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:01.964779 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ef657-5e2a-424a-8e20-9c82e9492aa8 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:01.974071 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1428/2827] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 08:39:04.156211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a58f-e865-4082-a915-4ca8318964a9 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:04.223803 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a58f-e865-4082-a915-4ca8318964a9 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:04.229632 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:04.230515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:04.230799 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:04.231074 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] PATH_INFO: `/v3/users/138942cb91954ab8acfe9e42f760fbc2` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:04.232389 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138942cb91954ab8acfe9e42f760fbc2)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:04.234341 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:04.239048 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:04.285822 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138942cb91954ab8acfe9e42f760fbc2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.293984 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138942cb91954ab8acfe9e42f760fbc2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.301483 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138942cb91954ab8acfe9e42f760fbc2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.302453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138942cb91954ab8acfe9e42f760fbc2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.314346 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138942cb91954ab8acfe9e42f760fbc2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.320580 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1400/2828] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:04 2022] DELETE /identity/v3/users/138942cb91954ab8acfe9e42f760fbc2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:04.325070 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9c78-5703-414e-bd78-b835e4f2f27d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:04.325600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9c78-5703-414e-bd78-b835e4f2f27d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:04.325600 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9c78-5703-414e-bd78-b835e4f2f27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:04.343177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-899c9c78-5703-414e-bd78-b835e4f2f27d None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:04.344404 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-899c9c78-5703-414e-bd78-b835e4f2f27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:04.407148 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1429/2829] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:04.417193 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:04.418740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:04.488573 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa6c63-1edb-4b0b-9e86-9d9c3e632807 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:04.493004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681466a5-f30c-4b87-a99f-e5341c751573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:04.493664 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681466a5-f30c-4b87-a99f-e5341c751573 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:04.493954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681466a5-f30c-4b87-a99f-e5341c751573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:04.494225 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681466a5-f30c-4b87-a99f-e5341c751573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:04.504777 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681466a5-f30c-4b87-a99f-e5341c751573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:04.506249 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681466a5-f30c-4b87-a99f-e5341c751573 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:04.516627 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1401/2830] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:04.798384 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9c78-5703-414e-bd78-b835e4f2f27d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:04.799944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-899c9c78-5703-414e-bd78-b835e4f2f27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:04.814259 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9c78-5703-414e-bd78-b835e4f2f27d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:04.820944 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:04.822124 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:04.822243 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:04.822514 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] PATH_INFO: `/v3/projects/76523ff80071488a8f4c0e7a4596cff5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:04.823812 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76523ff80071488a8f4c0e7a4596cff5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:04.825749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:04.879009 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76523ff80071488a8f4c0e7a4596cff5', 'request_id': 'req-a50928f8-a4b2-4bad-874f-a4006d8cdf53'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.884912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76523ff80071488a8f4c0e7a4596cff5', 'request_id': 'req-a50928f8-a4b2-4bad-874f-a4006d8cdf53'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.892646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76523ff80071488a8f4c0e7a4596cff5', 'request_id': 'req-a50928f8-a4b2-4bad-874f-a4006d8cdf53'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.903650 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] The token cache is being invalidate because project 76523ff80071488a8f4c0e7a4596cff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:04.903947 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.904604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:04.906161 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1430/2831] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:04 2022] DELETE /identity/v3/projects/76523ff80071488a8f4c0e7a4596cff5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:04.910331 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681466a5-f30c-4b87-a99f-e5341c751573 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:04.921036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681466a5-f30c-4b87-a99f-e5341c751573 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:04.991840 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:04.992731 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:04.993011 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:04.993285 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] PATH_INFO: `/v3/users/ac86ccb99b3747329d94fa39f3c5c9a9` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:04.994866 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac86ccb99b3747329d94fa39f3c5c9a9)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:04.996858 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:05.027700 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:05.054491 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac86ccb99b3747329d94fa39f3c5c9a9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.064072 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac86ccb99b3747329d94fa39f3c5c9a9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.072022 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac86ccb99b3747329d94fa39f3c5c9a9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.072686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac86ccb99b3747329d94fa39f3c5c9a9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.087051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac86ccb99b3747329d94fa39f3c5c9a9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.097402 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1402/2832] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:04 2022] DELETE /identity/v3/users/ac86ccb99b3747329d94fa39f3c5c9a9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:05.384150 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:05.385782 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:05.399466 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50928f8-a4b2-4bad-874f-a4006d8cdf53 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:05.447808 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:05.457841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:05.458921 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:05.459300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:05.459607 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] PATH_INFO: `/v3/projects/e894d8037e564371a6876173662960de` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:05.461056 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e894d8037e564371a6876173662960de)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:05.463228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:05.516303 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e894d8037e564371a6876173662960de', 'request_id': 'req-0ab605b6-9d37-4a6f-986c-169b3ff916c6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.521992 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e894d8037e564371a6876173662960de', 'request_id': 'req-0ab605b6-9d37-4a6f-986c-169b3ff916c6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.531416 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e894d8037e564371a6876173662960de', 'request_id': 'req-0ab605b6-9d37-4a6f-986c-169b3ff916c6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.542977 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] The token cache is being invalidate because project e894d8037e564371a6876173662960de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:05.543293 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.543907 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.545469 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1431/2833] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:05 2022] DELETE /identity/v3/projects/e894d8037e564371a6876173662960de => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:39:05.549503 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:05.550983 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:05.564396 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62e70b-d25c-4acb-94a6-4a6360a0a64a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:05.624541 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:05.625459 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:05.625736 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:05.626004 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] PATH_INFO: `/v3/users/0949acc349bb4212a51131c0b448c16f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:05.627371 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0949acc349bb4212a51131c0b448c16f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:05.629317 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:05.679540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0949acc349bb4212a51131c0b448c16f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.688100 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0949acc349bb4212a51131c0b448c16f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.695838 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0949acc349bb4212a51131c0b448c16f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.696570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0949acc349bb4212a51131c0b448c16f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.710713 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9a311577-2da5-4732-a4d5-0bdc52e9905f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0949acc349bb4212a51131c0b448c16f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:05.717219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1403/2834] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:05 2022] DELETE /identity/v3/users/0949acc349bb4212a51131c0b448c16f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:39:06.000613 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.002089 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:06.015375 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab605b6-9d37-4a6f-986c-169b3ff916c6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.071735 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.072605 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.072882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.073152 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] PATH_INFO: `/v3/projects/433cbef76a124245b55d84f146399348` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.074325 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=433cbef76a124245b55d84f146399348)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.076306 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.133714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '433cbef76a124245b55d84f146399348', 'request_id': 'req-30e2cccc-7d84-436b-b312-70555d8ef96e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:06.142446 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '433cbef76a124245b55d84f146399348', 'request_id': 'req-30e2cccc-7d84-436b-b312-70555d8ef96e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:06.153361 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '433cbef76a124245b55d84f146399348', 'request_id': 'req-30e2cccc-7d84-436b-b312-70555d8ef96e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:06.165345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] The token cache is being invalidate because project 433cbef76a124245b55d84f146399348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:06.165627 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:06.166256 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-30e2cccc-7d84-436b-b312-70555d8ef96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:06.167883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1432/2835] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:05 2022] DELETE /identity/v3/projects/433cbef76a124245b55d84f146399348 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:06.174103 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293840-4652-4118-8c48-c77517f59f02 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.174394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293840-4652-4118-8c48-c77517f59f02 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.174673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293840-4652-4118-8c48-c77517f59f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.194016 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-41293840-4652-4118-8c48-c77517f59f02 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:06.195239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-41293840-4652-4118-8c48-c77517f59f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:06.215159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:06.261310 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1404/2836] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:06.270826 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65836d2-93bf-49d2-834c-0b0f238bf7e0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.271109 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65836d2-93bf-49d2-834c-0b0f238bf7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.271413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65836d2-93bf-49d2-834c-0b0f238bf7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.289809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e65836d2-93bf-49d2-834c-0b0f238bf7e0 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:06.290995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e65836d2-93bf-49d2-834c-0b0f238bf7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:06.299838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1433/2837] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:06.303766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41293840-4652-4118-8c48-c77517f59f02 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.313394 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41293840-4652-4118-8c48-c77517f59f02 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.318565 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.319239 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.319596 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.319870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.320609 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:39:06.321131 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.323920 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.342780 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1405/2838] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 08:39:06.347662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65836d2-93bf-49d2-834c-0b0f238bf7e0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.357749 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65836d2-93bf-49d2-834c-0b0f238bf7e0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.363827 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121b386-ff20-43c3-8bc8-d4b1319b307a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.364490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121b386-ff20-43c3-8bc8-d4b1319b307a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.364786 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121b386-ff20-43c3-8bc8-d4b1319b307a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.365062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121b386-ff20-43c3-8bc8-d4b1319b307a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.365751 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121b386-ff20-43c3-8bc8-d4b1319b307a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.367714 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121b386-ff20-43c3-8bc8-d4b1319b307a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.371478 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:06.398226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1434/2839] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:39:06 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:06.402770 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.415699 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e3153-5d7f-4389-932b-10e1350f6134 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.474202 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40064a1-99ba-46d5-a73b-278fc082d791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.474873 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40064a1-99ba-46d5-a73b-278fc082d791 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.475162 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40064a1-99ba-46d5-a73b-278fc082d791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.475462 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40064a1-99ba-46d5-a73b-278fc082d791 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.476158 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40064a1-99ba-46d5-a73b-278fc082d791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.478105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40064a1-99ba-46d5-a73b-278fc082d791 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.499818 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1406/2840] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:39:06 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:06.503965 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121b386-ff20-43c3-8bc8-d4b1319b307a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.514841 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121b386-ff20-43c3-8bc8-d4b1319b307a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.519770 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eab3d7-d15f-4cd6-89bf-1b533726794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.520437 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eab3d7-d15f-4cd6-89bf-1b533726794c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.521059 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eab3d7-d15f-4cd6-89bf-1b533726794c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.521342 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eab3d7-d15f-4cd6-89bf-1b533726794c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.522076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eab3d7-d15f-4cd6-89bf-1b533726794c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.523733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eab3d7-d15f-4cd6-89bf-1b533726794c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.534086 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1435/2841] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:06.539252 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40064a1-99ba-46d5-a73b-278fc082d791 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.555759 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40064a1-99ba-46d5-a73b-278fc082d791 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.561617 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25862f-64ec-4f35-ae69-52a08bd535a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.562286 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25862f-64ec-4f35-ae69-52a08bd535a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.562568 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25862f-64ec-4f35-ae69-52a08bd535a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.562844 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25862f-64ec-4f35-ae69-52a08bd535a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.563612 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25862f-64ec-4f35-ae69-52a08bd535a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.565235 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25862f-64ec-4f35-ae69-52a08bd535a5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.576101 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1407/2842] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:06.582491 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eab3d7-d15f-4cd6-89bf-1b533726794c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.593934 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eab3d7-d15f-4cd6-89bf-1b533726794c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.600222 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee86d4-1156-4f9b-800e-144fedfde8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.601033 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee86d4-1156-4f9b-800e-144fedfde8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.601326 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee86d4-1156-4f9b-800e-144fedfde8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.601598 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee86d4-1156-4f9b-800e-144fedfde8c9 admin admin] PATH_INFO: `/v3/projects/88b94302ccb74a13ac098737091cf414/users/4a0bf065fe114d5a93c44e885ce46abf/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.617667 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee86d4-1156-4f9b-800e-144fedfde8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b94302ccb74a13ac098737091cf414, user_id=4a0bf065fe114d5a93c44e885ce46abf, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.620737 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee86d4-1156-4f9b-800e-144fedfde8c9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.635791 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1436/2843] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:06 2022] PUT /identity/v3/projects/88b94302ccb74a13ac098737091cf414/users/4a0bf065fe114d5a93c44e885ce46abf/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:06.647175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743af08-0931-4721-97e6-798b51a6056e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.647490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743af08-0931-4721-97e6-798b51a6056e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.647768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743af08-0931-4721-97e6-798b51a6056e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.674886 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d743af08-0931-4721-97e6-798b51a6056e None None] MFA Rules not processed for user `4a0bf065fe114d5a93c44e885ce46abf`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:06.676167 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d743af08-0931-4721-97e6-798b51a6056e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:06.743605 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1408/2844] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:06.748844 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee86d4-1156-4f9b-800e-144fedfde8c9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.816752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee86d4-1156-4f9b-800e-144fedfde8c9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.822570 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3caf47-7d11-41de-8e22-f77d3525876a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.823238 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3caf47-7d11-41de-8e22-f77d3525876a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.823545 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3caf47-7d11-41de-8e22-f77d3525876a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.823817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3caf47-7d11-41de-8e22-f77d3525876a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.824498 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3caf47-7d11-41de-8e22-f77d3525876a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.826442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3caf47-7d11-41de-8e22-f77d3525876a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.855929 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1437/2845] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:39:06 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:06.860160 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743af08-0931-4721-97e6-798b51a6056e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.874597 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743af08-0931-4721-97e6-798b51a6056e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.895222 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:06.939833 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28837459-b2d4-457d-bf96-9ee9f012cc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.940505 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837459-b2d4-457d-bf96-9ee9f012cc2e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.940787 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837459-b2d4-457d-bf96-9ee9f012cc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.941065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837459-b2d4-457d-bf96-9ee9f012cc2e admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.941756 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28837459-b2d4-457d-bf96-9ee9f012cc2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.943825 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28837459-b2d4-457d-bf96-9ee9f012cc2e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:06.964546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1409/2846] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:39:06 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:06.968693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3caf47-7d11-41de-8e22-f77d3525876a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:06.979489 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3caf47-7d11-41de-8e22-f77d3525876a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:06.985377 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ce8f4-93ce-4aa7-bb97-25dacece3025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:06.986039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ce8f4-93ce-4aa7-bb97-25dacece3025 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:06.986318 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ce8f4-93ce-4aa7-bb97-25dacece3025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:06.986587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ce8f4-93ce-4aa7-bb97-25dacece3025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:06.987341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ce8f4-93ce-4aa7-bb97-25dacece3025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:06.988961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ce8f4-93ce-4aa7-bb97-25dacece3025 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:07.000488 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1438/2847] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:07.005057 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28837459-b2d4-457d-bf96-9ee9f012cc2e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:07.015247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28837459-b2d4-457d-bf96-9ee9f012cc2e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:07.020721 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d4c79-44f3-4a4f-82a8-61b53d6bc418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:07.021624 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d4c79-44f3-4a4f-82a8-61b53d6bc418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:07.021905 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d4c79-44f3-4a4f-82a8-61b53d6bc418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:07.022181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d4c79-44f3-4a4f-82a8-61b53d6bc418 admin admin] PATH_INFO: `/v3/projects/aaba4c3c7a3f47ac9f35cfdaf000005f/users/93888f1b1fae48b2ba4cdb0aad11aa99/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:07.037857 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d4c79-44f3-4a4f-82a8-61b53d6bc418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaba4c3c7a3f47ac9f35cfdaf000005f, user_id=93888f1b1fae48b2ba4cdb0aad11aa99, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:07.040559 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d4c79-44f3-4a4f-82a8-61b53d6bc418 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:07.055556 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1410/2848] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:07 2022] PUT /identity/v3/projects/aaba4c3c7a3f47ac9f35cfdaf000005f/users/93888f1b1fae48b2ba4cdb0aad11aa99/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:07.059541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ce8f4-93ce-4aa7-bb97-25dacece3025 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:07.070448 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ce8f4-93ce-4aa7-bb97-25dacece3025 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:07.133428 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a9cda-2726-494f-a904-33ef5d5d9ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:07.134113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a9cda-2726-494f-a904-33ef5d5d9ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:07.134392 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a9cda-2726-494f-a904-33ef5d5d9ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:07.134660 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a9cda-2726-494f-a904-33ef5d5d9ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:07.135450 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a9cda-2726-494f-a904-33ef5d5d9ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:07.137236 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a9cda-2726-494f-a904-33ef5d5d9ee5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:07.150710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1439/2849] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:07.155023 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d4c79-44f3-4a4f-82a8-61b53d6bc418 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:07.166116 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d4c79-44f3-4a4f-82a8-61b53d6bc418 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:07.171545 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:07.172443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:07.172725 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:07.172999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] PATH_INFO: `/v3/projects/aaba4c3c7a3f47ac9f35cfdaf000005f/users/93888f1b1fae48b2ba4cdb0aad11aa99/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:07.175006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaba4c3c7a3f47ac9f35cfdaf000005f, user_id=93888f1b1fae48b2ba4cdb0aad11aa99, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:07.177792 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:07.188779 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1411/2850] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:07 2022] PUT /identity/v3/projects/aaba4c3c7a3f47ac9f35cfdaf000005f/users/93888f1b1fae48b2ba4cdb0aad11aa99/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:07.200034 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b6fc9-36e0-4caf-802a-bf754ed19ef2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:07.200382 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b6fc9-36e0-4caf-802a-bf754ed19ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:07.200724 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b6fc9-36e0-4caf-802a-bf754ed19ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:07.230693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a58b6fc9-36e0-4caf-802a-bf754ed19ef2 None None] MFA Rules not processed for user `93888f1b1fae48b2ba4cdb0aad11aa99`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:07.231963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a58b6fc9-36e0-4caf-802a-bf754ed19ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:07.240878 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:07.307126 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1440/2851] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:07.322409 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:07.325558 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:07.391046 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969e754-f1a3-47c6-80ad-b62c391366de admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:07.395025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decad95-6969-49ba-9fff-8f3370b556ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:07.395789 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decad95-6969-49ba-9fff-8f3370b556ec service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:07.396073 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decad95-6969-49ba-9fff-8f3370b556ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:07.396341 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decad95-6969-49ba-9fff-8f3370b556ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:07.406163 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decad95-6969-49ba-9fff-8f3370b556ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:07.407634 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decad95-6969-49ba-9fff-8f3370b556ec service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:07.417311 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1412/2852] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:07.524651 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b6fc9-36e0-4caf-802a-bf754ed19ef2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:07.541297 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b6fc9-36e0-4caf-802a-bf754ed19ef2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:07.548281 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec503e-5b1e-4e54-95d8-61c61b341b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:07.549159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec503e-5b1e-4e54-95d8-61c61b341b78 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:07.549547 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec503e-5b1e-4e54-95d8-61c61b341b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:07.549942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec503e-5b1e-4e54-95d8-61c61b341b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:07.614795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec503e-5b1e-4e54-95d8-61c61b341b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:07.616145 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec503e-5b1e-4e54-95d8-61c61b341b78 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:07.625700 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1441/2853] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:10.222351 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decad95-6969-49ba-9fff-8f3370b556ec service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:10.304957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decad95-6969-49ba-9fff-8f3370b556ec service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:10.313637 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:10.314518 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:10.314795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:10.315108 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] PATH_INFO: `/v3/users/4a0bf065fe114d5a93c44e885ce46abf` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:10.316502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0bf065fe114d5a93c44e885ce46abf)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:10.318571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:10.338159 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:10.381265 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0bf065fe114d5a93c44e885ce46abf'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:10.392006 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0bf065fe114d5a93c44e885ce46abf'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:10.399691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0bf065fe114d5a93c44e885ce46abf'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:10.400401 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0bf065fe114d5a93c44e885ce46abf'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:10.413065 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0bf065fe114d5a93c44e885ce46abf'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:10.422870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1413/2854] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:10 2022] DELETE /identity/v3/users/4a0bf065fe114d5a93c44e885ce46abf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:10.428464 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e78c5-b01f-49ac-bc69-9ae823dde9b6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:10.428957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e78c5-b01f-49ac-bc69-9ae823dde9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:10.428957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e78c5-b01f-49ac-bc69-9ae823dde9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:10.447643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-005e78c5-b01f-49ac-bc69-9ae823dde9b6 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:10.448942 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-005e78c5-b01f-49ac-bc69-9ae823dde9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:10.521832 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1442/2855] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:39:10.532325 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:10.533837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:10.609448 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9e35d-3c7d-4097-9531-756d74956247 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:10.614210 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813f7a4-9895-4476-b99c-600dc5ac3375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:10.614898 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813f7a4-9895-4476-b99c-600dc5ac3375 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:10.615179 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813f7a4-9895-4476-b99c-600dc5ac3375 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:10.615484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813f7a4-9895-4476-b99c-600dc5ac3375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:10.637005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813f7a4-9895-4476-b99c-600dc5ac3375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:10.638512 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813f7a4-9895-4476-b99c-600dc5ac3375 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:10.648965 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1414/2856] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:10.935251 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e78c5-b01f-49ac-bc69-9ae823dde9b6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:10.936741 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-005e78c5-b01f-49ac-bc69-9ae823dde9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:10.950912 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e78c5-b01f-49ac-bc69-9ae823dde9b6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:10.955595 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:10.956479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:10.956763 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:10.957566 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] PATH_INFO: `/v3/projects/88b94302ccb74a13ac098737091cf414` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:10.958914 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b94302ccb74a13ac098737091cf414)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:10.961228 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:11.005953 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:11.013898 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b94302ccb74a13ac098737091cf414', 'request_id': 'req-03fb02de-df5b-486b-a069-d68a227839c2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.019327 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b94302ccb74a13ac098737091cf414', 'request_id': 'req-03fb02de-df5b-486b-a069-d68a227839c2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.027935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b94302ccb74a13ac098737091cf414', 'request_id': 'req-03fb02de-df5b-486b-a069-d68a227839c2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.039270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] The token cache is being invalidate because project 88b94302ccb74a13ac098737091cf414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:11.039559 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.040267 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.041857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1443/2857] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:10 2022] DELETE /identity/v3/projects/88b94302ccb74a13ac098737091cf414 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:39:11.046106 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813f7a4-9895-4476-b99c-600dc5ac3375 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:11.056910 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813f7a4-9895-4476-b99c-600dc5ac3375 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:11.115757 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:11.116695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:11.116970 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:11.117237 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] PATH_INFO: `/v3/users/93888f1b1fae48b2ba4cdb0aad11aa99` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:11.118436 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93888f1b1fae48b2ba4cdb0aad11aa99)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:11.120737 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:11.171287 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:11.180291 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93888f1b1fae48b2ba4cdb0aad11aa99'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.188952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93888f1b1fae48b2ba4cdb0aad11aa99'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.196096 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93888f1b1fae48b2ba4cdb0aad11aa99'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.196743 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93888f1b1fae48b2ba4cdb0aad11aa99'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.209036 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1e42b520-f5d6-4005-9c2e-3be4a672a517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93888f1b1fae48b2ba4cdb0aad11aa99'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.215359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1415/2858] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:11 2022] DELETE /identity/v3/users/93888f1b1fae48b2ba4cdb0aad11aa99 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:39:11.493230 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:11.495231 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:11.514234 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb02de-df5b-486b-a069-d68a227839c2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:11.570528 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:11.571343 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:11.571624 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:11.571899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] PATH_INFO: `/v3/projects/aaba4c3c7a3f47ac9f35cfdaf000005f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:11.573076 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaba4c3c7a3f47ac9f35cfdaf000005f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:11.574996 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:11.625995 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaba4c3c7a3f47ac9f35cfdaf000005f', 'request_id': 'req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.632074 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaba4c3c7a3f47ac9f35cfdaf000005f', 'request_id': 'req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.641632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaba4c3c7a3f47ac9f35cfdaf000005f', 'request_id': 'req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.653176 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] The token cache is being invalidate because project aaba4c3c7a3f47ac9f35cfdaf000005f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:11.653462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.654099 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a9842a5a-c7ba-4061-b0c1-8c33ba993eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:11.655695 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1444/2859] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:11 2022] DELETE /identity/v3/projects/aaba4c3c7a3f47ac9f35cfdaf000005f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:11.660719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4360be9-356e-4237-9abd-50f1f29c43ab None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:11.661211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4360be9-356e-4237-9abd-50f1f29c43ab None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:11.661322 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4360be9-356e-4237-9abd-50f1f29c43ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:11.680356 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e4360be9-356e-4237-9abd-50f1f29c43ab None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:11.681570 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e4360be9-356e-4237-9abd-50f1f29c43ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:11.744907 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1416/2860] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:11.758048 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de38c-ccad-4af1-be70-4bf459a84bf8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:11.758338 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de38c-ccad-4af1-be70-4bf459a84bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:11.758620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de38c-ccad-4af1-be70-4bf459a84bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:11.779116 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5c3de38c-ccad-4af1-be70-4bf459a84bf8 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:11.780356 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5c3de38c-ccad-4af1-be70-4bf459a84bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:11.788962 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1445/2861] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:11.793369 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4360be9-356e-4237-9abd-50f1f29c43ab None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:11.803796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4360be9-356e-4237-9abd-50f1f29c43ab None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:11.810195 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:11.810859 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:11.811150 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:11.811464 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:11.812207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:39:11.812517 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:11.814139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:11.827104 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1417/2862] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:39:11.831370 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3de38c-ccad-4af1-be70-4bf459a84bf8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:11.840976 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3de38c-ccad-4af1-be70-4bf459a84bf8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:11.845932 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebda96-d867-4bbc-b6b0-0a5d4cc09952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:11.846587 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebda96-d867-4bbc-b6b0-0a5d4cc09952 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:11.846870 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebda96-d867-4bbc-b6b0-0a5d4cc09952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:11.847143 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebda96-d867-4bbc-b6b0-0a5d4cc09952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:11.847869 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebda96-d867-4bbc-b6b0-0a5d4cc09952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:11.849783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebda96-d867-4bbc-b6b0-0a5d4cc09952 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:11.878988 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1446/2863] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:39:11 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 08:39:11.883305 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:11.893623 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df1c1d-7f96-498e-b298-01a7883acdca admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:11.966467 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aeff80-2c39-40ea-9b14-4262b33bd37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:11.967125 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aeff80-2c39-40ea-9b14-4262b33bd37e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:11.967439 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aeff80-2c39-40ea-9b14-4262b33bd37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:11.967693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aeff80-2c39-40ea-9b14-4262b33bd37e admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:11.968393 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aeff80-2c39-40ea-9b14-4262b33bd37e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:11.970301 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aeff80-2c39-40ea-9b14-4262b33bd37e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:11.993945 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1418/2864] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:39:11 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:11.998541 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebda96-d867-4bbc-b6b0-0a5d4cc09952 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.013534 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebda96-d867-4bbc-b6b0-0a5d4cc09952 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.020113 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7078-b2be-443f-999f-54dfc2efb8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.020778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7078-b2be-443f-999f-54dfc2efb8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.021092 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7078-b2be-443f-999f-54dfc2efb8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.021367 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7078-b2be-443f-999f-54dfc2efb8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.022097 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be7078-b2be-443f-999f-54dfc2efb8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.024336 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be7078-b2be-443f-999f-54dfc2efb8fc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.036039 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1447/2865] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:39:12.041089 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aeff80-2c39-40ea-9b14-4262b33bd37e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.051571 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aeff80-2c39-40ea-9b14-4262b33bd37e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.057082 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ed11a-5b98-431f-9c14-40b8c650fdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.057727 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ed11a-5b98-431f-9c14-40b8c650fdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.058000 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ed11a-5b98-431f-9c14-40b8c650fdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.058274 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ed11a-5b98-431f-9c14-40b8c650fdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.058975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ed11a-5b98-431f-9c14-40b8c650fdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.060615 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ed11a-5b98-431f-9c14-40b8c650fdc1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.070976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1419/2866] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 08:39:12.077294 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7078-b2be-443f-999f-54dfc2efb8fc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.092694 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7078-b2be-443f-999f-54dfc2efb8fc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.101495 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69befc47-1185-4dae-8e18-07fc93b56ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.102395 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befc47-1185-4dae-8e18-07fc93b56ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.102681 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befc47-1185-4dae-8e18-07fc93b56ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.102956 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befc47-1185-4dae-8e18-07fc93b56ea1 admin admin] PATH_INFO: `/v3/projects/8f0ebe429d35426ea866893304f9da5d/users/55eb624bdae5455e89709095c3f4719b/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.119959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69befc47-1185-4dae-8e18-07fc93b56ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0ebe429d35426ea866893304f9da5d, user_id=55eb624bdae5455e89709095c3f4719b, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.122630 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69befc47-1185-4dae-8e18-07fc93b56ea1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.124926 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:12.139531 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1448/2867] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:12 2022] PUT /identity/v3/projects/8f0ebe429d35426ea866893304f9da5d/users/55eb624bdae5455e89709095c3f4719b/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:12.149977 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a0a7-22fe-472c-9719-809827e43d78 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.150267 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a0a7-22fe-472c-9719-809827e43d78 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.150546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a0a7-22fe-472c-9719-809827e43d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.184229 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2fe1a0a7-22fe-472c-9719-809827e43d78 None None] MFA Rules not processed for user `55eb624bdae5455e89709095c3f4719b`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:12.185458 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2fe1a0a7-22fe-472c-9719-809827e43d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:12.196451 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:12.255514 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1420/2868] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:12.260310 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69befc47-1185-4dae-8e18-07fc93b56ea1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.325322 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69befc47-1185-4dae-8e18-07fc93b56ea1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.330778 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0274cc-6ef9-4a3d-87eb-18526fb1ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.331409 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0274cc-6ef9-4a3d-87eb-18526fb1ed33 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.331693 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0274cc-6ef9-4a3d-87eb-18526fb1ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.331961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0274cc-6ef9-4a3d-87eb-18526fb1ed33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.332646 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0274cc-6ef9-4a3d-87eb-18526fb1ed33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.334579 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0274cc-6ef9-4a3d-87eb-18526fb1ed33 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.362662 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1449/2869] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:39:12 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:12.368641 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1a0a7-22fe-472c-9719-809827e43d78 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.379996 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1a0a7-22fe-472c-9719-809827e43d78 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.438987 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caca13-fdd5-43b5-8042-79017d41836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.439724 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caca13-fdd5-43b5-8042-79017d41836f admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.440003 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caca13-fdd5-43b5-8042-79017d41836f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.440278 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caca13-fdd5-43b5-8042-79017d41836f admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.440975 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caca13-fdd5-43b5-8042-79017d41836f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.442914 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caca13-fdd5-43b5-8042-79017d41836f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.464168 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1421/2870] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:39:12 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:12.468295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0274cc-6ef9-4a3d-87eb-18526fb1ed33 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.478806 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0274cc-6ef9-4a3d-87eb-18526fb1ed33 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.484313 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207148a-af5b-46e1-90ca-3afd13113518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.485005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207148a-af5b-46e1-90ca-3afd13113518 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.485270 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207148a-af5b-46e1-90ca-3afd13113518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.485548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207148a-af5b-46e1-90ca-3afd13113518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.486248 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207148a-af5b-46e1-90ca-3afd13113518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.487940 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207148a-af5b-46e1-90ca-3afd13113518 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.500032 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1450/2871] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:12.504324 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caca13-fdd5-43b5-8042-79017d41836f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.514853 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caca13-fdd5-43b5-8042-79017d41836f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.519726 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bbb6e-e70e-4850-a7d5-884d20f6cf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.520628 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bbb6e-e70e-4850-a7d5-884d20f6cf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.520909 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bbb6e-e70e-4850-a7d5-884d20f6cf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.521183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bbb6e-e70e-4850-a7d5-884d20f6cf1e admin admin] PATH_INFO: `/v3/projects/cc5a3801cd3c49549fdac92705f737bc/users/7a2528e7f7ea411da376111bfb4a5f1d/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.537025 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89bbb6e-e70e-4850-a7d5-884d20f6cf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5a3801cd3c49549fdac92705f737bc, user_id=7a2528e7f7ea411da376111bfb4a5f1d, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.539954 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89bbb6e-e70e-4850-a7d5-884d20f6cf1e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.555863 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1422/2872] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:12 2022] PUT /identity/v3/projects/cc5a3801cd3c49549fdac92705f737bc/users/7a2528e7f7ea411da376111bfb4a5f1d/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:12.560209 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207148a-af5b-46e1-90ca-3afd13113518 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.570410 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207148a-af5b-46e1-90ca-3afd13113518 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.632880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350262a-02cf-4b10-ab8c-8b4124855610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.634052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350262a-02cf-4b10-ab8c-8b4124855610 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.634549 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350262a-02cf-4b10-ab8c-8b4124855610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.635062 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350262a-02cf-4b10-ab8c-8b4124855610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.635961 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350262a-02cf-4b10-ab8c-8b4124855610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.637801 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350262a-02cf-4b10-ab8c-8b4124855610 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.649501 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1451/2873] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:12.654502 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bbb6e-e70e-4850-a7d5-884d20f6cf1e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.666043 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bbb6e-e70e-4850-a7d5-884d20f6cf1e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.672300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.673211 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.673515 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.673788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] PATH_INFO: `/v3/projects/cc5a3801cd3c49549fdac92705f737bc/users/7a2528e7f7ea411da376111bfb4a5f1d/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.675967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5a3801cd3c49549fdac92705f737bc, user_id=7a2528e7f7ea411da376111bfb4a5f1d, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.678613 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.690200 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1423/2874] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:12 2022] PUT /identity/v3/projects/cc5a3801cd3c49549fdac92705f737bc/users/7a2528e7f7ea411da376111bfb4a5f1d/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:12.701724 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d2dda-587a-47c3-99c9-fcbbecf2f9eb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.702044 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d2dda-587a-47c3-99c9-fcbbecf2f9eb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.702345 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d2dda-587a-47c3-99c9-fcbbecf2f9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.730957 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d90d2dda-587a-47c3-99c9-fcbbecf2f9eb None None] MFA Rules not processed for user `7a2528e7f7ea411da376111bfb4a5f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:12.732207 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d90d2dda-587a-47c3-99c9-fcbbecf2f9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:12.793486 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:12.814284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1452/2875] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:12.826999 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:12.829188 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:12.892492 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069493-ec93-48d3-b793-e86eda895e36 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:12.896561 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a23d9-3cce-4f66-b75c-b2d269c7e8c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:12.897207 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a23d9-3cce-4f66-b75c-b2d269c7e8c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:12.897490 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a23d9-3cce-4f66-b75c-b2d269c7e8c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:12.897766 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a23d9-3cce-4f66-b75c-b2d269c7e8c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:12.918227 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:12.960960 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a23d9-3cce-4f66-b75c-b2d269c7e8c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:12.962524 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a23d9-3cce-4f66-b75c-b2d269c7e8c9 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:12.971884 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1424/2876] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:17.301963 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d2dda-587a-47c3-99c9-fcbbecf2f9eb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:17.304111 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d90d2dda-587a-47c3-99c9-fcbbecf2f9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:17.319853 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d2dda-587a-47c3-99c9-fcbbecf2f9eb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:17.324659 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80253966-2414-4828-b292-12c6ce523877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:17.325302 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253966-2414-4828-b292-12c6ce523877 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:17.325576 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253966-2414-4828-b292-12c6ce523877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:17.325846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253966-2414-4828-b292-12c6ce523877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:17.337164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80253966-2414-4828-b292-12c6ce523877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:17.338900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80253966-2414-4828-b292-12c6ce523877 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:17.348883 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1453/2877] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:22.323551 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a23d9-3cce-4f66-b75c-b2d269c7e8c9 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:22.399376 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a23d9-3cce-4f66-b75c-b2d269c7e8c9 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:22.405279 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:22.406165 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:22.406443 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:22.406715 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] PATH_INFO: `/v3/users/55eb624bdae5455e89709095c3f4719b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:22.408083 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=55eb624bdae5455e89709095c3f4719b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:22.410051 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:22.459427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55eb624bdae5455e89709095c3f4719b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:22.468695 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55eb624bdae5455e89709095c3f4719b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:22.475967 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55eb624bdae5455e89709095c3f4719b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:22.476796 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55eb624bdae5455e89709095c3f4719b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:22.495603 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55eb624bdae5455e89709095c3f4719b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:22.502942 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1425/2878] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:22 2022] DELETE /identity/v3/users/55eb624bdae5455e89709095c3f4719b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:22.507987 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758167-e9e8-4467-b39b-0c46fb91adf3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:22.508522 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758167-e9e8-4467-b39b-0c46fb91adf3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:22.508631 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758167-e9e8-4467-b39b-0c46fb91adf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:22.528519 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc758167-e9e8-4467-b39b-0c46fb91adf3 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:22.529726 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc758167-e9e8-4467-b39b-0c46fb91adf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:22.591341 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1454/2879] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:22.606958 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:22.609629 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:22.657343 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:22.676521 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de78ce-b8c1-4cb9-9f39-a87c212ae7ef admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:22.680175 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a13ca-981e-4496-b4d3-dc4b88b645a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:22.680837 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a13ca-981e-4496-b4d3-dc4b88b645a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:22.681121 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a13ca-981e-4496-b4d3-dc4b88b645a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:22.681400 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a13ca-981e-4496-b4d3-dc4b88b645a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:22.691185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a13ca-981e-4496-b4d3-dc4b88b645a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:22.692666 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a13ca-981e-4496-b4d3-dc4b88b645a0 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:22.702932 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1426/2880] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:22.988874 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc758167-e9e8-4467-b39b-0c46fb91adf3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:22.990479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc758167-e9e8-4467-b39b-0c46fb91adf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:23.004632 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc758167-e9e8-4467-b39b-0c46fb91adf3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:23.010906 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:23.012586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:23.012586 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:23.012919 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] PATH_INFO: `/v3/projects/8f0ebe429d35426ea866893304f9da5d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:23.013889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0ebe429d35426ea866893304f9da5d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:23.015850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:23.065101 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0ebe429d35426ea866893304f9da5d', 'request_id': 'req-9efa1a2f-544b-4289-bd19-5af3e27d34b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.070548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0ebe429d35426ea866893304f9da5d', 'request_id': 'req-9efa1a2f-544b-4289-bd19-5af3e27d34b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.080227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0ebe429d35426ea866893304f9da5d', 'request_id': 'req-9efa1a2f-544b-4289-bd19-5af3e27d34b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.091636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] The token cache is being invalidate because project 8f0ebe429d35426ea866893304f9da5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:23.091943 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.092604 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.094164 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1455/2881] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:22 2022] DELETE /identity/v3/projects/8f0ebe429d35426ea866893304f9da5d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:23.098452 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a13ca-981e-4496-b4d3-dc4b88b645a0 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:23.108691 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a13ca-981e-4496-b4d3-dc4b88b645a0 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:23.173670 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:23.174593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:23.174868 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:23.175139 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] PATH_INFO: `/v3/users/7a2528e7f7ea411da376111bfb4a5f1d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:23.176472 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a2528e7f7ea411da376111bfb4a5f1d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:23.178405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:23.233245 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a2528e7f7ea411da376111bfb4a5f1d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.242593 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a2528e7f7ea411da376111bfb4a5f1d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.250875 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a2528e7f7ea411da376111bfb4a5f1d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.251546 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a2528e7f7ea411da376111bfb4a5f1d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.264716 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-655cc015-70de-4d29-9ebc-4de79fed73e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a2528e7f7ea411da376111bfb4a5f1d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.271037 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1427/2882] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:23 2022] DELETE /identity/v3/users/7a2528e7f7ea411da376111bfb4a5f1d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:23.551849 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:23.553284 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:23.567548 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa1a2f-544b-4289-bd19-5af3e27d34b6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:23.633597 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:23.634507 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:23.634938 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:23.635339 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] PATH_INFO: `/v3/projects/cc5a3801cd3c49549fdac92705f737bc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:23.637745 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc5a3801cd3c49549fdac92705f737bc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:23.640846 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:23.643159 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:23.691005 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc5a3801cd3c49549fdac92705f737bc', 'request_id': 'req-c8450351-269a-42c3-a5b8-082b44cce18f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.696817 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc5a3801cd3c49549fdac92705f737bc', 'request_id': 'req-c8450351-269a-42c3-a5b8-082b44cce18f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.707046 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc5a3801cd3c49549fdac92705f737bc', 'request_id': 'req-c8450351-269a-42c3-a5b8-082b44cce18f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.718177 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] The token cache is being invalidate because project cc5a3801cd3c49549fdac92705f737bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:23.718463 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.719093 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c8450351-269a-42c3-a5b8-082b44cce18f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:23.720672 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1456/2883] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:23 2022] DELETE /identity/v3/projects/cc5a3801cd3c49549fdac92705f737bc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:23.726788 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de359a7-fe35-430a-adf4-45d64aefcafb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:23.727453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de359a7-fe35-430a-adf4-45d64aefcafb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:23.727453 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de359a7-fe35-430a-adf4-45d64aefcafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:23.747044 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2de359a7-fe35-430a-adf4-45d64aefcafb None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:23.748296 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2de359a7-fe35-430a-adf4-45d64aefcafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:23.806055 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:23.818183 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1428/2884] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 08:39:23.829353 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aa62c-9714-404b-a039-84c4a4fde7bf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:23.829643 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aa62c-9714-404b-a039-84c4a4fde7bf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:23.829954 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aa62c-9714-404b-a039-84c4a4fde7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:23.848620 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f61aa62c-9714-404b-a039-84c4a4fde7bf None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:23.849865 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f61aa62c-9714-404b-a039-84c4a4fde7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:23.859479 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1457/2885] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:23.864484 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de359a7-fe35-430a-adf4-45d64aefcafb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:23.877870 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de359a7-fe35-430a-adf4-45d64aefcafb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:23.883823 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:23.884411 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:23.884686 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:23.884957 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:23.885673 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 08:39:23.885976 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:23.891592 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:23.901185 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1429/2886] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 08:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 08:39:23.906300 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aa62c-9714-404b-a039-84c4a4fde7bf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:23.920175 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aa62c-9714-404b-a039-84c4a4fde7bf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:23.925920 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c492fa9-22a2-4dda-9210-e6abea65e053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:23.926574 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492fa9-22a2-4dda-9210-e6abea65e053 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:23.926857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492fa9-22a2-4dda-9210-e6abea65e053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:23.927130 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492fa9-22a2-4dda-9210-e6abea65e053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:23.927850 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c492fa9-22a2-4dda-9210-e6abea65e053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:23.929781 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c492fa9-22a2-4dda-9210-e6abea65e053 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:23.958103 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1458/2887] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:39:23 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:23.964433 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:23.979712 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3c6e2-d4fb-44a7-a6f7-e5a986aede40 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.039319 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb36ca-1f21-4752-afb3-9ed1b9865923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.039989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb36ca-1f21-4752-afb3-9ed1b9865923 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.040273 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb36ca-1f21-4752-afb3-9ed1b9865923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.040542 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb36ca-1f21-4752-afb3-9ed1b9865923 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.041247 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb36ca-1f21-4752-afb3-9ed1b9865923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.043281 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb36ca-1f21-4752-afb3-9ed1b9865923 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.065169 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1430/2888] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:39:23 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:24.069451 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c492fa9-22a2-4dda-9210-e6abea65e053 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.080413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c492fa9-22a2-4dda-9210-e6abea65e053 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.085133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5a1a-7a7f-47ec-bcff-644927577fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.085790 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5a1a-7a7f-47ec-bcff-644927577fae admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.086069 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5a1a-7a7f-47ec-bcff-644927577fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.086402 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5a1a-7a7f-47ec-bcff-644927577fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.087129 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a5a1a-7a7f-47ec-bcff-644927577fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.088777 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a5a1a-7a7f-47ec-bcff-644927577fae admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.101250 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1459/2889] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:24.105427 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb36ca-1f21-4752-afb3-9ed1b9865923 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.116602 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb36ca-1f21-4752-afb3-9ed1b9865923 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.122300 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f57cbe-4451-44ef-9f78-c95119e07d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.123017 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f57cbe-4451-44ef-9f78-c95119e07d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.123330 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f57cbe-4451-44ef-9f78-c95119e07d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.123623 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f57cbe-4451-44ef-9f78-c95119e07d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.124359 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f57cbe-4451-44ef-9f78-c95119e07d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.126019 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f57cbe-4451-44ef-9f78-c95119e07d4c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.136186 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1431/2890] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:24.140584 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5a1a-7a7f-47ec-bcff-644927577fae admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.151225 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5a1a-7a7f-47ec-bcff-644927577fae admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.156295 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468db640-9476-41ff-87c1-6f0387558e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.157172 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468db640-9476-41ff-87c1-6f0387558e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.157440 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468db640-9476-41ff-87c1-6f0387558e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.157717 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468db640-9476-41ff-87c1-6f0387558e44 admin admin] PATH_INFO: `/v3/projects/d6042b4806854fa786676fee1022ae6c/users/93195d7724a741d08710b0a12e4028e5/roles/af1e89bc75e74c9f819959d898d086a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.173237 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468db640-9476-41ff-87c1-6f0387558e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6042b4806854fa786676fee1022ae6c, user_id=93195d7724a741d08710b0a12e4028e5, role_id=af1e89bc75e74c9f819959d898d086a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.176036 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468db640-9476-41ff-87c1-6f0387558e44 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.191941 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1460/2891] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:24 2022] PUT /identity/v3/projects/d6042b4806854fa786676fee1022ae6c/users/93195d7724a741d08710b0a12e4028e5/roles/af1e89bc75e74c9f819959d898d086a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:24.203827 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2b7bd-5334-441c-8232-e0c08fa6fcdb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.204124 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2b7bd-5334-441c-8232-e0c08fa6fcdb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.204405 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2b7bd-5334-441c-8232-e0c08fa6fcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.234555 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-89c2b7bd-5334-441c-8232-e0c08fa6fcdb None None] MFA Rules not processed for user `93195d7724a741d08710b0a12e4028e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:24.235816 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-89c2b7bd-5334-441c-8232-e0c08fa6fcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:24.314280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1432/2892] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:24.318989 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468db640-9476-41ff-87c1-6f0387558e44 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.385592 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468db640-9476-41ff-87c1-6f0387558e44 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.391021 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da876be-f705-4c9f-9338-6ad0ff5e00fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.391635 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da876be-f705-4c9f-9338-6ad0ff5e00fa admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.391915 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da876be-f705-4c9f-9338-6ad0ff5e00fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.392168 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da876be-f705-4c9f-9338-6ad0ff5e00fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.392857 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da876be-f705-4c9f-9338-6ad0ff5e00fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.394766 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da876be-f705-4c9f-9338-6ad0ff5e00fa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.424935 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1461/2893] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 08:39:24 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:24.430456 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2b7bd-5334-441c-8232-e0c08fa6fcdb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.441522 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2b7bd-5334-441c-8232-e0c08fa6fcdb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.498399 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:24.506952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932e18d-b5f3-4693-9f0b-b54d1f435836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.507861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932e18d-b5f3-4693-9f0b-b54d1f435836 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.508536 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932e18d-b5f3-4693-9f0b-b54d1f435836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.508880 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932e18d-b5f3-4693-9f0b-b54d1f435836 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.509719 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932e18d-b5f3-4693-9f0b-b54d1f435836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.512176 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932e18d-b5f3-4693-9f0b-b54d1f435836 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.538887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1433/2894] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 08:39:24 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 08:39:24.542959 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da876be-f705-4c9f-9338-6ad0ff5e00fa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.553710 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da876be-f705-4c9f-9338-6ad0ff5e00fa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.559900 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137480da-8f74-4803-8441-af02b361671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.560560 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137480da-8f74-4803-8441-af02b361671e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.560838 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137480da-8f74-4803-8441-af02b361671e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.561105 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137480da-8f74-4803-8441-af02b361671e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.561816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137480da-8f74-4803-8441-af02b361671e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.563462 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137480da-8f74-4803-8441-af02b361671e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.568889 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:24.576513 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1462/2895] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:24.579733 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932e18d-b5f3-4693-9f0b-b54d1f435836 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.590123 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932e18d-b5f3-4693-9f0b-b54d1f435836 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.595334 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e466a24-64ab-423e-88bd-8e996aca8352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.596212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e466a24-64ab-423e-88bd-8e996aca8352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.596494 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e466a24-64ab-423e-88bd-8e996aca8352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.596769 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e466a24-64ab-423e-88bd-8e996aca8352 admin admin] PATH_INFO: `/v3/projects/d85de167dd154253b889bee79caad4fc/users/1e93366a08c84ff7986ac5d118c061e5/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.612919 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e466a24-64ab-423e-88bd-8e996aca8352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d85de167dd154253b889bee79caad4fc, user_id=1e93366a08c84ff7986ac5d118c061e5, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.615653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e466a24-64ab-423e-88bd-8e996aca8352 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.636574 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1434/2896] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:24 2022] PUT /identity/v3/projects/d85de167dd154253b889bee79caad4fc/users/1e93366a08c84ff7986ac5d118c061e5/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:24.640666 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137480da-8f74-4803-8441-af02b361671e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.651249 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137480da-8f74-4803-8441-af02b361671e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.710133 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a183c61-dc2e-4631-be73-73bd79c47580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.710802 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183c61-dc2e-4631-be73-73bd79c47580 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.711080 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183c61-dc2e-4631-be73-73bd79c47580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.711399 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183c61-dc2e-4631-be73-73bd79c47580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.712125 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a183c61-dc2e-4631-be73-73bd79c47580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.713733 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a183c61-dc2e-4631-be73-73bd79c47580 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.724618 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1463/2897] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 08:39:24.731085 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e466a24-64ab-423e-88bd-8e996aca8352 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.742218 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e466a24-64ab-423e-88bd-8e996aca8352 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.748349 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.749280 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.749563 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.749836 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] PATH_INFO: `/v3/projects/d85de167dd154253b889bee79caad4fc/users/1e93366a08c84ff7986ac5d118c061e5/roles/26c4193120bd43c69b4c3756d41b3949` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.751952 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d85de167dd154253b889bee79caad4fc, user_id=1e93366a08c84ff7986ac5d118c061e5, role_id=26c4193120bd43c69b4c3756d41b3949)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:24.754638 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:24.765406 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1435/2898] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 08:39:24 2022] PUT /identity/v3/projects/d85de167dd154253b889bee79caad4fc/users/1e93366a08c84ff7986ac5d118c061e5/roles/26c4193120bd43c69b4c3756d41b3949 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:24.776809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bcfef-d660-4bbe-83f4-28d99ff727bb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.777096 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bcfef-d660-4bbe-83f4-28d99ff727bb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.777366 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bcfef-d660-4bbe-83f4-28d99ff727bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:24.804634 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c47bcfef-d660-4bbe-83f4-28d99ff727bb None None] MFA Rules not processed for user `1e93366a08c84ff7986ac5d118c061e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:24.805820 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c47bcfef-d660-4bbe-83f4-28d99ff727bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:24.877603 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1464/2899] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:24.889212 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:24.891180 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:24.952540 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0f656-9d3e-4a21-b642-a8110ad9191a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:24.957097 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cf6b1-3764-4e28-831e-c6c8a8ea85f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '419321fbd62c4d95a39cfd498b4a97d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:24.957750 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cf6b1-3764-4e28-831e-c6c8a8ea85f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:24.958028 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cf6b1-3764-4e28-831e-c6c8a8ea85f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:24.958323 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cf6b1-3764-4e28-831e-c6c8a8ea85f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:25.016883 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cf6b1-3764-4e28-831e-c6c8a8ea85f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:25.018348 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cf6b1-3764-4e28-831e-c6c8a8ea85f3 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:25.027234 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1436/2900] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:27.588049 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bcfef-d660-4bbe-83f4-28d99ff727bb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:27.652816 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bcfef-d660-4bbe-83f4-28d99ff727bb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:27.658035 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:27.658910 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:27.659188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:27.659490 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] PATH_INFO: `/v3/users/93195d7724a741d08710b0a12e4028e5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:27.660698 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93195d7724a741d08710b0a12e4028e5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:27.662636 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:27.685261 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:27.715052 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93195d7724a741d08710b0a12e4028e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:27.715752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93195d7724a741d08710b0a12e4028e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:27.723188 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93195d7724a741d08710b0a12e4028e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:27.736254 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93195d7724a741d08710b0a12e4028e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:27.745594 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93195d7724a741d08710b0a12e4028e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:27.751784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1465/2901] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:27 2022] DELETE /identity/v3/users/93195d7724a741d08710b0a12e4028e5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:27.756404 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181ecca-bcda-4c5c-b49c-fead6f323811 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:27.756887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181ecca-bcda-4c5c-b49c-fead6f323811 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:27.756992 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181ecca-bcda-4c5c-b49c-fead6f323811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:27.774795 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c181ecca-bcda-4c5c-b49c-fead6f323811 None None] MFA Rules not processed for user `76b6ddf40257432aa78fadccd4348cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 08:39:27.775948 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c181ecca-bcda-4c5c-b49c-fead6f323811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:27.837653 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1437/2902] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 08:39:27.850902 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:27.853194 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:27.910081 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75bbf-7e74-4d88-a861-59f20ce61894 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:27.916842 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6f1c-daf9-4189-8e86-0ee2583b1a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2965b1c8ab52457f8f167acda4ba6844', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:27.917809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6f1c-daf9-4189-8e86-0ee2583b1a56 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:27.918226 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6f1c-daf9-4189-8e86-0ee2583b1a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:27.918591 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6f1c-daf9-4189-8e86-0ee2583b1a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:27.932899 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e6f1c-daf9-4189-8e86-0ee2583b1a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:27.935182 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e6f1c-daf9-4189-8e86-0ee2583b1a56 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:27.946544 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1466/2903] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:39:28.233989 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181ecca-bcda-4c5c-b49c-fead6f323811 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:28.235693 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c181ecca-bcda-4c5c-b49c-fead6f323811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:28.245861 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:28.254516 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181ecca-bcda-4c5c-b49c-fead6f323811 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:28.260005 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:28.260860 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:28.261156 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:28.261481 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] PATH_INFO: `/v3/projects/d6042b4806854fa786676fee1022ae6c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:28.262887 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6042b4806854fa786676fee1022ae6c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:28.265091 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:28.316372 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6042b4806854fa786676fee1022ae6c', 'request_id': 'req-229888e5-1506-473b-8434-1ef71eb61f4e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.322102 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6042b4806854fa786676fee1022ae6c', 'request_id': 'req-229888e5-1506-473b-8434-1ef71eb61f4e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.330995 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6042b4806854fa786676fee1022ae6c', 'request_id': 'req-229888e5-1506-473b-8434-1ef71eb61f4e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.342181 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] The token cache is being invalidate because project d6042b4806854fa786676fee1022ae6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:28.342468 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.343105 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.344752 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1438/2904] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:28 2022] DELETE /identity/v3/projects/d6042b4806854fa786676fee1022ae6c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:28.348829 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6f1c-daf9-4189-8e86-0ee2583b1a56 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:28.358435 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6f1c-daf9-4189-8e86-0ee2583b1a56 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:28.417796 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:28.418784 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:28.419134 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:28.419442 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] PATH_INFO: `/v3/users/1e93366a08c84ff7986ac5d118c061e5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:28.420809 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e93366a08c84ff7986ac5d118c061e5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:28.423312 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:28.472880 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:28.474800 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e93366a08c84ff7986ac5d118c061e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.475612 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e93366a08c84ff7986ac5d118c061e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.483948 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e93366a08c84ff7986ac5d118c061e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.496413 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e93366a08c84ff7986ac5d118c061e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.505242 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e93366a08c84ff7986ac5d118c061e5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.511485 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1467/2905] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 08:39:28 2022] DELETE /identity/v3/users/1e93366a08c84ff7986ac5d118c061e5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 08:39:28.801208 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:39:28.802696 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:39:28.816095 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229888e5-1506-473b-8434-1ef71eb61f4e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:39:28.879039 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b6ddf40257432aa78fadccd4348cd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44e27258dd6f419b9b00f5d07551d7c9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:39:28.880029 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:39:28.880315 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:39:28.880591 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] PATH_INFO: `/v3/projects/d85de167dd154253b889bee79caad4fc` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:39:28.881768 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d85de167dd154253b889bee79caad4fc)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:39:28.883740 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:39:28.950392 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd85de167dd154253b889bee79caad4fc', 'request_id': 'req-c6d72857-a7f5-45df-bdbf-d3a597ddb690'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.959856 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd85de167dd154253b889bee79caad4fc', 'request_id': 'req-c6d72857-a7f5-45df-bdbf-d3a597ddb690'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.972352 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd85de167dd154253b889bee79caad4fc', 'request_id': 'req-c6d72857-a7f5-45df-bdbf-d3a597ddb690'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.984219 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] The token cache is being invalidate because project d85de167dd154253b889bee79caad4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 08:39:28.984508 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.985177 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c6d72857-a7f5-45df-bdbf-d3a597ddb690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 08:39:28.986804 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1439/2906] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 08:39:28 2022] DELETE /identity/v3/projects/d85de167dd154253b889bee79caad4fc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 08:39:38.507882 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=13,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:39:38.981263 dv-cinder-847216-2-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 08:43:46.813945 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 08:43:46.815795 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:43:46.985752 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb57e9-8c0e-4237-b276-837543a87504 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 08:43:47.047568 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d374-6a58-46b6-acc9-5fae92a2381b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93c07dd32f543a69e74599297cbe17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '772001415dd543878fb9fd8e3fe599f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 08:43:47.048227 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d374-6a58-46b6-acc9-5fae92a2381b service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 08:43:47.048508 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d374-6a58-46b6-acc9-5fae92a2381b service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 08:43:47.048783 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d374-6a58-46b6-acc9-5fae92a2381b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 08:43:47.050747 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d676d374-6a58-46b6-acc9-5fae92a2381b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 08:43:47.062320 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d374-6a58-46b6-acc9-5fae92a2381b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 08:43:47.063886 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d374-6a58-46b6-acc9-5fae92a2381b service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 08:43:47.073971 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1468/2907] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 08:43:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 08:43:57.069638 dv-cinder-847216-2-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}